A structured L-BFGS method with diagonal scaling and its application to image registration

TL;DR

This paper introduces a structured L-BFGS optimization method with diagonal scaling tailored for inverse problems like image registration, improving convergence and performance by exploiting problem structure.

Contribution

It proposes a novel structured L-BFGS method using diagonal scaling for the seed matrix, enhancing efficiency in non-convex inverse problems.

Findings

Boosts performance on real-life image registration tasks

Ensures global and linear convergence on non-convex problems

Provides a freely available implementation

Abstract

We devise an L-BFGS method for optimization problems in which the objective is the sum of two functions, where the Hessian of the first function is computationally unavailable while the Hessian of the second function has a computationally available approximation that allows for cheap matrix-vector products. This is a prototypical setting for many inverse problems. The proposed L-BFGS method exploits the structure of the objective to construct a more accurate Hessian approximation than in standard L-BFGS. In contrast to existing works on structured L-BFGS, we choose the first part of the seed matrix, which approximates the Hessian of the first function, as a diagonal matrix rather than a multiple of the identity. We derive two suitable formulas for the coefficients of the diagonal matrix and show that this boosts performance on real-life image registration problems, which are highly non-convex inverse problems. The new method converges globally and linearly on non-convex problems under mild assumptions in a general Hilbert space setting, making it applicable to a broad class of inverse problems. An implementation of the method is freely available.