Newton-type Alternating Minimization Algorithm for Convex Optimization

TL;DR

The paper introduces NAMA, a Newton-type alternating minimization algorithm for structured nonsmooth convex optimization, which enhances convergence speed and is suitable for large-scale problems through quasi-Newton directions.

Contribution

It presents a novel line-search method based on the dual problem's penalty function, achieving superlinear convergence with quasi-Newton directions, and demonstrates improved performance over existing methods.

Findings

NAMA exhibits strong convergence properties.

Superlinear convergence achieved with quasi-Newton directions.

Limited-memory directions significantly improve convergence speed.

Abstract

We propose NAMA (Newton-type Alternating Minimization Algorithm) for solving structured nonsmooth convex optimization problems where the sum of two functions is to be minimized, one being strongly convex and the other composed with a linear mapping. The proposed algorithm is a line-search method over a continuous, real-valued, exact penalty function for the corresponding dual problem, which is computed by evaluating the augmented Lagrangian at the primal points obtained by alternating minimizations. As a consequence, NAMA relies on exactly the same computations as the classical alternating minimization algorithm (AMA), also known as the dual proximal gradient method. Under standard assumptions the proposed algorithm possesses strong convergence properties, while under mild additional assumptions the asymptotic convergence is superlinear, provided that the search directions are chosen according to quasi-Newton formulas. Due to its simplicity, the proposed method is well suited for embedded applications and large-scale problems. Experiments show that using limited-memory directions in NAMA greatly improves the convergence speed over AMA and its accelerated variant.