泰勒展开与梯度下降

从一阶泰勒展开推导梯度下降

一阶泰勒展开：

$$f\left(x\right)\approx f\left(x_0\right)+\left(x-x_0\right)f’\left(x_0\right)$$ $$\begin{gathered} f(x+ϵ)=f(x)+ϵf^{\prime }(x) \\ ϵ=-\eta f^{\prime }(x) \end{gathered}$$ $$\begin{gathered} f(x-\eta f^{\prime }(x))=f(x)+(\eta f^{\prime }(x))f^{\prime }(x) \\ =f(x)-\eta f^{\prime }(x)<f(x) \end{gathered}$$

这样要使得$f(x)$变小，只需要每次更新：$x=x-\eta f^{\prime }(x)$，当然$\eta$要很小