Proximal Newton-type methods for minimizing composite functions

TL;DR

This paper extends Newton-type optimization methods to handle composite functions involving both smooth and nonsmooth convex components, ensuring convergence even with inexact computations, and unifies several existing methods under this framework.

Contribution

It introduces a generalized proximal Newton-type framework for composite convex functions, providing new convergence guarantees and unifying various existing algorithms.

Findings

Proximal Newton methods inherit convergence properties of classical Newton methods.

The methods converge even with inexact search directions.

Several bioinformatics, signal processing, and statistical learning algorithms are special cases.

Abstract

We generalize Newton-type methods for minimizing smooth functions to handle a sum of two convex functions: a smooth function and a nonsmooth function with a simple proximal mapping. We show that the resulting proximal Newton-type methods inherit the desirable convergence behavior of Newton-type methods for minimizing smooth functions, even when search directions are computed inexactly. Many popular methods tailored to problems arising in bioinformatics, signal processing, and statistical learning are special cases of proximal Newton-type methods, and our analysis yields new convergence results for some of these methods.