Affine-Invariant Global Non-Asymptotic Convergence Analysis of BFGS under Self-Concordance

TL;DR

This paper proves that BFGS, a popular optimization algorithm, converges globally and efficiently for a broad class of functions called strongly self-concordant, with guarantees that are invariant under affine transformations.

Contribution

The paper provides the first affine-invariant, global non-asymptotic convergence analysis of BFGS under the assumption of strong self-concordance, relaxing traditional convexity and smoothness conditions.

Findings

BFGS achieves global linear and superlinear convergence under self-concordance.

Convergence guarantees depend only on initial error and self-concordant constant.

Results extend BFGS theory beyond classical assumptions.

Abstract

In this paper, we establish global non-asymptotic convergence guarantees for the BFGS quasi-Newton method without requiring strong convexity or the Lipschitz continuity of the gradient or Hessian. Instead, we consider the setting where the objective function is strictly convex and strongly self-concordant. For an arbitrary initial point and any arbitrary positive-definite initial Hessian approximation, we prove global linear and superlinear convergence guarantees for BFGS when the step size is determined using a line search scheme satisfying the weak Wolfe conditions. Moreover, all our global guarantees are affine-invariant, with the convergence rates depending solely on the initial error and the strongly self-concordant constant. Our results extend the global non-asymptotic convergence theory of BFGS beyond traditional assumptions and, for the first time, establish affine-invariant convergence guarantees aligning with the inherent affine invariance of the BFGS method.