Forward-backward truncated Newton methods for convex composite optimization

TL;DR

This paper introduces two proximal Newton-CG algorithms for convex nonsmooth composite optimization, leveraging the forward-backward envelope to enable efficient Newton-type methods with fast convergence.

Contribution

The paper develops novel proximal Newton-CG methods based on the forward-backward envelope, combining line search and efficiency estimates for improved optimization performance.

Findings

Algorithms achieve fast asymptotic convergence

Each iteration involves solving small linear systems

Methods are computationally attractive

Abstract

This paper proposes two proximal Newton-CG methods for convex nonsmooth optimization problems in composite form. The algorithms are based on a a reformulation of the original nonsmooth problem as the unconstrained minimization of a continuously differentiable function, namely the forward-backward envelope (FBE). The first algorithm is based on a standard line search strategy, whereas the second one combines the global efficiency estimates of the corresponding first-order methods, while achieving fast asymptotic convergence rates. Furthermore, they are computationally attractive since each Newton iteration requires the approximate solution of a linear system of usually small dimension.