The divergence of the BFGS and Gauss Newton Methods

TL;DR

This paper constructs specific examples demonstrating divergence of BFGS and Gauss Newton methods, even under conditions like bounded level sets and convexity along search lines, highlighting limitations of these optimization algorithms.

Contribution

It provides new explicit examples of divergence for BFGS and Gauss Newton methods under conditions similar to recent published examples, extending understanding of their limitations.

Findings

Examples show divergence despite bounded level sets

Iterates and gradients fit previous divergence framework

Highlights limitations of BFGS and Gauss Newton methods

Abstract

We present examples of divergence for the BFGS and Gauss Newton methods. These examples have objective functions with bounded level sets and other properties concerning the examples published recently in this journal, like unit steps and convexity along the search lines. As these other examples, the iterates, function values and gradients in the new examples fit into the general formulation in our previous work {\it On the divergence of line search methods, Comput. Appl. Math. vol.26 no.1 (2007)}, which also presents an example of divergence for Newton's method.