On a fixed-point continuation method for a convex optimization problem

TL;DR

This paper introduces a fixed-point continuation method for convex optimization that efficiently computes solutions across a range of penalty parameters, with proven convergence and convergence rate, and explores its relation to epsilon-subdifferential-based algorithms.

Contribution

The paper proposes a novel fixed-point continuation algorithm for convex optimization that approximates the entire trade-off curve and establishes its convergence properties.

Findings

Algorithm converges with proven rate

Computes solutions for multiple penalty parameters simultaneously

Shows relation to epsilon-subdifferential iterative algorithms

Abstract

We consider a variation of the classical proximal-gradient algorithm for the iterative minimization of a cost function consisting of a sum of two terms, one smooth and the other prox-simple, and whose relative weight is determined by a penalty parameter. This so-called fixed-point continuation method allows one to approximate the problem's trade-off curve, i.e. to compute the minimizers of the cost function for a whole range of values of the penalty parameter at once. The algorithm is shown to converge, and a rate of convergence of the cost function is also derived. Furthermore, it is shown that this method is related to iterative algorithms constructed on the basis of the $\epsilon$-subdifferential of the prox-simple term. Some numerical examples are provided.