A structured L-BFGS method and its application to inverse problems

TL;DR

This paper introduces a structured L-BFGS optimization method tailored for inverse problems, demonstrating global convergence and superior performance on non-convex, real-world applications like medical image registration.

Contribution

It develops a novel structured L-BFGS algorithm with proven convergence in non-convex settings and applies it effectively to inverse problems.

Findings

The method converges globally without convexity assumptions.

It outperforms other structured and classical L-BFGS methods on real and model problems.

It demonstrates linear convergence near strongly convex cluster points.

Abstract

Many inverse problems are phrased as optimization problems in which the objective function is the sum of a data-fidelity term and a regularization. Often, the Hessian of the fidelity term is computationally unavailable while the Hessian of the regularizer allows for cheap matrix-vector products. In this paper, we study an LBFGS method that takes advantage of this structure. We show that the method converges globally without convexity assumptions and that the convergence is linear under a Kurdyka--{\L}ojasiewicz-type inequality. In addition, we prove linear convergence to cluster points near which the objective function is strongly convex. To the best of our knowledge, this is the first time that linear convergence of an LBFGS method is established in a non-convex setting. The convergence analysis is carried out in infinite dimensional Hilbert space, which is appropriate for inverse problems but has not been done before. Numerical results show that the new method outperforms other structured LBFGS methods and classical LBFGS on non-convex real-life problems from medical image registration. It also compares favorably with classical LBFGS on ill-conditioned quadratic model problems. An implementation of the method is freely available.