On Quasi-Newton Forward--Backward Splitting: Proximal Calculus and Convergence

TL;DR

This paper develops a new quasi-Newton forward-backward splitting framework with a special metric that enables efficient proximal mapping calculations, leading to accelerated convex optimization algorithms with proven convergence.

Contribution

It introduces a novel proximal calculus in a diagonal plus rank-r metric, facilitating efficient computations and convergence proofs for quasi-Newton splitting methods.

Findings

Efficient proximal mapping evaluation via duality and piece-wise linearity.

The proposed method converges and outperforms existing algorithms in numerical tests.

Applicable to diverse fields like signal processing, machine learning, and sparse recovery.

Abstract

We introduce a framework for quasi-Newton forward--backward splitting algorithms (proximal quasi-Newton methods) with a metric induced by diagonal $\pm$ rank-$r$ symmetric positive definite matrices. This special type of metric allows for a highly efficient evaluation of the proximal mapping. The key to this efficiency is a general proximal calculus in the new metric. By using duality, formulas are derived that relate the proximal mapping in a rank-$r$ modified metric to the original metric. We also describe efficient implementations of the proximity calculation for a large class of functions; the implementations exploit the piece-wise linear nature of the dual problem. Then, we apply these results to acceleration of composite convex minimization problems, which leads to elegant quasi-Newton methods for which we prove convergence. The algorithm is tested on several numerical examples and compared to a comprehensive list of alternatives in the literature. Our quasi-Newton splitting algorithm with the prescribed metric compares favorably against state-of-the-art. The algorithm has extensive applications including signal processing, sparse recovery, machine learning and classification to name a few.