Newton's Method

Proof Technique
Newton's Method is a method of solving an equation for which there is no convenient closed form solution.

The derivative of the equation has to be known in order to use Newton's Method.

Let:
 * the equation to be solved be of the form:
 * $y = f \left({x}\right)$


 * the value of $x$ is required for a given $y$.

Then an iterative improvement on an initial guess is of the form
 * $x_2 = x_1 - \dfrac {f \left({x_1}\right) -y} {f' \left({x_1}\right)}$

where $f' \left({x}\right)$ is the derivative of $f$ evaluated at $x$.

Proof
The function $f \left({x}\right)$ can be expanded using Taylor's Theorem:


 * $f \left({x_2}\right) = f \left({x_1}\right) + f' \left({x_1}\right) \left({x_2 - x_1}\right) + \dfrac 1 2 f'' \left({x_1}\right) \left({x_2 - x_1}\right)^2 + \dotsb$

As $x_2$ gets closer to $x_1$, this series can be truncated to:
 * $f \left({x_2}\right) = f \left({x_1}\right) + f' \left({x_1}\right) \left({x_2 - x_1}\right)$

Let $x_\infty$ be the exact solution where:
 * $f \left({x_\infty}\right) = y$

Let $\epsilon \in \R_{>0}$ be the difference from the new estimate to the solution:
 * $x_\infty = x_2 + \epsilon$

Then the function expanded around the new estimate is:
 * $y = f \left({x_2}\right) + f' \left({x_2}\right) \epsilon$

Solving for $x_2$ produces:
 * $x_2 = x_1 - \dfrac {f \left({x_1}\right) - y} {f' \left({x_1}\right)} - \epsilon \dfrac {f' \left({x_2}\right)} {f' \left({x_1}\right)}$

For $\epsilon$ small enough the final term can be neglected:
 * $x_2 = x_1 - \dfrac {f \left({x_1}\right) - y} {f' \left({x_1}\right)}$