On the convergence of BFGS on a class of piecewise linear non-smooth functions

TL;DR

This paper proves that the BFGS algorithm terminates finitely on certain unbounded convex piecewise linear functions, generating unbounded directions and avoiding convergence to non-stationary points, thus extending its theoretical understanding.

Contribution

It provides the first theoretical convergence analysis of BFGS on a class of non-smooth, unbounded convex piecewise linear functions.

Findings

BFGS terminates finitely on the considered functions.

The algorithm generates an unbounded direction.

It does not converge to a non-stationary point.

Abstract

The quasi-Newton Broyden-Fletcher-Goldfarb-Shanno (BFGS) method has proven to be very reliable and efficient for the minimization of smooth objective functions since its inception in the 1960s. Recently, it was observed empirically that it also works remarkably well for non-smooth problems when combined with the Armijo-Wolfe line search, but only very limited theoretical convergence theory could be established so far. In this paper, we expand these results by considering convex piecewise linear functions with two pieces that are not bounded below. We prove that the algorithm always terminates in a finite number of iterations, eventually generating an unbounded direction. In other words, in contrast to the gradient method, the BFGS algorithm does not converge to a non-stationary point.