Subspace decomposition in regularized least-squares: solution properties, restricted coercivity and beyond

TL;DR

This paper analyzes the solution properties of regularized least-squares problems using subspace decomposition, introducing concepts like restricted coercivity and linking existing results through conjugate functions.

Contribution

It develops a unified framework for understanding solution existence, uniqueness, and compactness in regularized least-squares via subspace decomposition and conjugate functions.

Findings

Derived explicit solution set expressions using conjugate functions

Unified various existing results on solution properties

Introduced the concept of restricted coercivity and linked it to recession cones

Abstract

We investigate the solution properties of the regularized least-squares problem. Using a subspace decomposition technique, we derive expressions for the solution set in terms of the conjugate function, from which various properties, including existence, compactness and uniqueness, can then be easily analyzed. A key distinction of our approach from existing works is the separate treatment of existence and compactness. We unify many existing results based on recession cones and sublevel sets, and link them to our findings by connecting the recession function with the recession cone of the subdifferential of the conjugate function. In particular, the concept of restricted coercivity is developed and discussed in various aspects. The associated linearly constrained counterpart is discussed in a similar manner. Its connections to regularized least-squares are further established via the exactness of infimal postcomposition. Our results are supported by numerous examples, among which the geometric interpretation of the lasso solution deserves further investigations in near future.