Nonsmooth Newton methods with effective subspaces for polyhedral regularization

TL;DR

This paper introduces new nonsmooth Newton methods for convex optimization with polyhedral regularizers, achieving quadratic convergence and demonstrating broad applicability and accelerated performance in various problems.

Contribution

The paper develops several novel nonsmooth Newton algorithms that avoid complex second-order computations and attain quadratic convergence under tilt-stability.

Findings

Achieved quadratic convergence rates in experiments.

Demonstrated broad applicability across multiple problem types.

Outperformed existing first-order and recent nonsmooth Newton methods.

Abstract

We propose several new nonsmooth Newton methods for solving convex composite optimization problems with polyhedral regularizers, while avoiding the computation of complicated second-order information on these functions. Under the tilt-stability condition at the optimal solution, these methods achieve the quadratic convergence rates expected of Newton schemes. Numerical experiments on Lasso, generalized Lasso, OSCAR-regularized least-square problems, and an image super-resolution task illustrate both the broad applicability and the accelerated convergence profile of the proposed algorithms, in comparison with first-order and several recently developed nonsmooth Newton schemes.