我编写了一些代码,对几个数据点执行梯度下降。 由于某种原因,曲线没有正确收敛,但我不知道为什么会这样。我总是以爆炸的尾部告终。
我是否做错了其中一项计算?我实际上陷入了局部最小值还是其他原因?
这是我的代码:
import numpy as np
import matplotlib.pyplot as plt
def estimate(weights, x, order):
est = 0
for i in range(order):
est += weights[i] * x ** i
return est
def cost_function(x, y, weights, m):
cost = 0
for i in range(m-1):
cost += (((weights[i] * x ** i) - y) ** 2)
return (np.sum(cost ** 2) / ( 2 * m ))
def descent(A, b, iterations, descent_rate, order):
x = A.T[0]
y = b.reshape(4)
# features
ones = np.vstack(np.ones(len(A)))
x = np.vstack(A.T[0])
x2 = np.vstack(A.T[0] ** 2)
# Our feature matrix
features = np.concatenate((ones,x,x2), axis = 1).T
# Initialize our coefficients to zero
weights = np.zeros(order + 1)
m = len(y)
# gradient descent
for i in range(iterations):
est = estimate(weights, x, order).T
difference = est - y
weights = weights + (-descent_rate * (1/m) * np.matmul(difference, features.T)[0])
cost = cost_function(x, y, weights, m)
print(cost)
plt.scatter(x,y)
u = np.linspace(0,3,100)
plt.plot(u, (u ** 2) * weights[2] + u * weights[1] + weights[0], '-')
plt.show()
A = np.array(((0,1),
(1,1),
(2,1),
(3,1)))
b = np.array((1,2,0,3), ndmin = 2 ).T
iterations = 150
descent_rate = 0.01
order = 2
descent(A, b, iterations, descent_rate, order)
我希望避免陷入这样的最低限度。我尝试将初始权重设置为随机值,但无济于事,有时它会下降一点,但随后会再次出现相同的行为。
这是通过最小二乘解获得的预期结果:
最佳答案
您的估计
函数应该是
def estimate(weights, x, order):
est = 0
for i in range(order+1):
est += weights[i] * x ** i
return est
更好的是,由于顺序信息已经存在于 权重
向量的大小中,因此可以使用以下方法删除冗余:
def estimate(weights, x):
est = 0
for i in range(len(weights)):
est += weights[i] * x ** i
return est
关于python - 梯度下降曲线拟合,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/56209615/