An active-set method for sparse approximations. Part I: Separable $\ell_1$ terms

TL;DR

This paper introduces an active-set method combining proximal multipliers and semismooth Newton techniques for efficient sparse approximation in convex quadratic problems, demonstrating superior convergence and practical performance.

Contribution

It presents a novel active-set algorithm integrating PMM and SSN methods with Krylov solvers and preconditioning, improving efficiency and convergence in sparse convex quadratic optimization.

Findings

Achieves global convergence under feasibility assumptions.

Demonstrates local superlinear convergence of the SSN scheme.

Outperforms existing solvers like OSQP and IP-PMM in numerical tests.

Abstract

In this paper we present an active-set method for the solution of $\ell_1$-regularized convex quadratic optimization problems. It is derived by combining a proximal method of multipliers (PMM) strategy with a standard semismooth Newton method (SSN). The resulting linear systems are solved using a Krylov-subspace method, accelerated by certain general-purpose preconditioners which are shown to be optimal with respect to the proximal parameters. Practical efficiency is further improved by warm-starting the algorithm using a proximal alternating direction method of multipliers. We show that the outer PMM achieves global convergence under mere feasibility assumptions. Under additional standard assumptions, the PMM scheme achieves global linear and local superlinear convergence. The SSN scheme is locally superlinearly convergent, assuming that its associated linear systems are solved accurately enough, and globally convergent under certain additional regularity assumptions. We provide numerical evidence to demonstrate the effectiveness of the approach by comparing it against OSQP and IP-PMM (an ADMM and a regularized IPM solver, respectively) on several elastic-net linear regression and $L^1$-regularized PDE-constrained optimization problems.