Geometry of Sparsity-Inducing Norms

TL;DR

This paper explores the geometric structure of sparsity-inducing norms that explicitly promote solutions with a fixed number of nonzero entries, providing conditions and properties that facilitate support identification in sparse optimization.

Contribution

It introduces generalized $k$-support dual norms for controlling sparsity, analyzes their geometric properties, and characterizes the faces of their unit balls, advancing sparse optimization theory.

Findings

Conditions under which $k$-support dual norms promote $k$-sparse solutions

Structural characterization of unit ball faces as hypersimplices

Geometric analysis of $k$-support dual norms for $ ext{l}_p$-norms

Abstract

Sparse optimization seeks an optimal solution with few nonzero entries. To achieve this, it is common to add to the criterion a penalty term proportional to the $\ell_1$-norm, which is recognized as the archetype of sparsity-inducing norms. In this approach, the number of nonzero entries is not controlled a priori. By contrast, in this paper, our motivation is to find an optimal solution with at most~$k$ nonzero coordinates (or for short, $k$-sparse vectors), where $k$ is a given sparsity threshold (or ``sparsity budget''). For this purpose, we study the class of generalized $k$-support dual~norms that arise from any given so-called source norm. When added as a penalty term, we provide conditions under which such generalized $k$-support dual~norms promote $k$-sparse solutions. The result follows from an analysis of the exposed faces of closed convex sets generated by $k$-sparse vectors, and of how primal support identification can be deduced from dual information. Finally, we study some of the geometric properties of the unit balls for the $k$-support dual~norms and their dual norms when the source norm belongs to the family of $\ell_p$-norms. In particular, we show a striking structural property: every proper face of the unit balls for the $k$-support dual~norms is a hypersimplex, i.e., the convex hull of $0/1$-valued points with the same $\ell_0$-norm.