A Hybrid Quasi-Newton Projected-Gradient Method with Application to Lasso and Basis-Pursuit Denoise

TL;DR

This paper introduces a hybrid quasi-Newton projected-gradient algorithm for convex optimization over polyhedral sets, demonstrating its effectiveness on Lasso and basis-pursuit denoise problems with proven convergence.

Contribution

It extends spectral projected-gradient methods with limited-memory BFGS, providing a new algorithm for convex optimization on polyhedral sets with convergence guarantees.

Findings

Effective for Lasso and basis-pursuit denoise problems

Converges under suitable conditions

Suitable for simple domains and bound-constrained problems

Abstract

We propose a new algorithm for the optimization of convex functions over a polyhedral set in Rn. The algorithm extends the spectral projected-gradient method with limited-memory BFGS iterates restricted to the present face whenever possible. We prove convergence of the algorithm under suitable conditions and apply the algorithm to solve the Lasso problem, and consequently, the basis-pursuit denoise problem through the root-finding framework proposed by van den Berg and Friedlander [SIAM Journal on Scientific Computing, 31(2), 2008]. The algorithm is especially well suited to simple domains and could also be used to solve bound-constrained problems as well as problems restricted to the simplex.