Adaptive FISTA for Non-convex Optimization

TL;DR

This paper introduces an adaptive FISTA variant that optimizes the extrapolation parameter via line search, providing convergence guarantees for non-convex problems and demonstrating improved efficiency in inverse problems.

Contribution

It develops an adaptively extrapolated proximal gradient method with convergence guarantees for non-convex optimization, linking it to proximal quasi-Newton methods.

Findings

Proposed method converges in non-convex settings.

Algorithm shows improved efficiency in numerical experiments.

Establishes new convergence guarantees for proximal quasi-Newton methods.

Abstract

In this paper we propose an adaptively extrapolated proximal gradient method, which is based on the accelerated proximal gradient method (also known as FISTA), however we locally optimize the extrapolation parameter by carrying out an exact (or inexact) line search. It turns out that in some situations, the proposed algorithm is equivalent to a class of SR1 (identity minus rank 1) proximal quasi-Newton methods. Convergence is proved in a general non-convex setting, and hence, as a byproduct, we also obtain new convergence guarantees for proximal quasi-Newton methods. The efficiency of the new method is shown in numerical experiments on a sparsity regularized non-linear inverse problem.