Globally Convergent Inexact Newton Methods

• Published in: SIAM journal on optimization Vol. 4; no. 2; pp. 393 - 422
• Main Authors: Eisenstat, Stanley C., Walker, Homer F.
• Format: Journal Article
• Language: English
• Published: Philadelphia Society for Industrial and Applied Mathematics 01.05.1994
• Subjects: Algorithms; Globalization; Iterative methods; Methods; Optimization
• ISSN: 1052-6234; 1095-7189
• DOI: 10.1137/0804022

Inexact Newton methods for finding a zero of $F:\mathbf{R}^n \to \mathbf{R}^n $ are variations of Newton's method in which each step only approximately satisfies the linear Newton equation but still reduces the norm of the local linear model of $F$. Here, inexact Newton methods are formulated that incorporate features designed to improve convergence from arbitrary starting points. For each method, a basic global convergence result is established to the effect that, under reasonable assumptions, if a sequence of iterates has a limit point at which $F^\prime $ is invertible, then that limit point is a solution and the sequence converges to it. When appropriate, it is shown that initial inexact Newton steps are taken near the solution, and so the convergence can ultimately be made as fast as desired, up to the rate of Newton's method, by forcing the initial linear residuals to be appropriately small. The primary goal is to introduce and analyze new inexact Newton methods, but consideration is also given to "globalizations" of (exact) Newton's method that can naturally be viewed as inexact Newton methods.