Lipschitz continuity of solution multifunctions of extended $\ell_1$ regularization problems

TL;DR

This paper studies the Lipschitz continuity of solution multifunctions in extended ll_1 regularization problems like Lasso, providing conditions for continuity and single-valuedness under parameter perturbations.

Contribution

It offers new characterizations of Lipschitz continuity for solution multifunctions in regularization problems, including cases with data matrix perturbations.

Findings

Lipschitz continuity of solutions relative to regularization and observation parameters.

Full characterization of solution continuity when data matrix also perturbs.

Lipschitz continuity implies the solution is single-valued for Lasso.

Abstract

The Lasso and the basis pursuit in compressed sensing and machine learning are convex optimization problems with three parameters: the regularization scalar, the observation vector and the data matrix. Relative to the first two parameters, we obtain the Lipschitz continuity of the solution multifunction on its convex domain. When the data matrix of the Lasso also perturbs, where non-polyhedral structure may display, we obtain full characterizations for the Lipschitz continuity of the solution multifunction on the product of a compact and convex set in the space of first two parameters and a neighborhood of the fixed data matrix. Moreover for the solution multifunction of the Lasso, we show that the Lipschitz continuity implies its single-valuedness. Our analysis is based on polyhedron theory, a sufficient condition that ensures the Lipschitz continuity of a polyhedral multifunction with a convex domain, and an explicit representation of the solution multifunction, where the latter is a consequence of the Lipschitz continuity of the solution multifunction relative to the first two parameters.