An efficient active-set method with applications to sparse approximations and risk minimization

TL;DR

This paper introduces a new active-set method combining proximal multipliers and semismooth Newton techniques, optimized with novel preconditioners, to efficiently solve large-scale convex quadratic programs with applications in finance, machine learning, and PDEs.

Contribution

The paper develops a globally convergent active-set algorithm that integrates PMM and SSN, with innovative preconditioners for Krylov methods, enhancing scalability and efficiency in nonsmooth convex quadratic programming.

Findings

Demonstrates superior performance on real-world datasets

Shows robustness across diverse applications

Achieves faster convergence compared to existing methods

Abstract

In this paper we present an efficient active-set method for the solution of convex quadratic programming problems with general piecewise-linear terms in the objective, with applications to sparse approximations and risk-minimization. The algorithm is derived by combining a proximal method of multipliers (PMM) with a standard semismooth Newton method (SSN), and is shown to be globally convergent under minimal assumptions. Further local linear (and potentially superlinear) convergence is shown under standard additional conditions. The major computational bottleneck of the proposed approach arises from the solution of the associated SSN linear systems. These are solved using a Krylov-subspace method, accelerated by certain novel general-purpose preconditioners which are shown to be optimal with respect to the proximal penalty parameters. The preconditioners are easy to store and invert, since they exploit the structure of the nonsmooth terms appearing in the problem's objective to significantly reduce their memory requirements. We showcase the efficiency, robustness, and scalability of the proposed solver on a variety of problems arising in risk-averse portfolio selection, $L^1$-regularized partial differential equation constrained optimization, quantile regression, and binary classification via linear support vector machines. We provide computational evidence, on real-world datasets, to demonstrate the ability of the solver to efficiently and competitively handle a diverse set of medium- and large-scale optimization instances.