The Geometry of Uniqueness, Sparsity and Clustering in Penalized Estimation

TL;DR

This paper establishes a geometric condition for the uniqueness of penalized least-squares estimators with polytope-norm penalties, covering various methods like LASSO and SLOPE, and explores their sparsity and clustering properties.

Contribution

It provides a necessary and sufficient geometric condition for estimator uniqueness and introduces the concept of SLOPE patterns to describe sparsity and clustering features.

Findings

Uniqueness condition involves the intersection of the design matrix row span with faces of the dual norm unit ball.

Characterization of SLOPE patterns explains sparsity and clustering behavior.

Geometric insights apply to a broad class of penalized estimation methods.

Abstract

We provide a necessary and sufficient condition for the uniqueness of penalized least-squares estimators whose penalty term is given by a norm with a polytope unit ball, covering a wide range of methods including SLOPE, PACS, fused, clustered and classical LASSO as well as the related method of basis pursuit. We consider a strong type of uniqueness that is relevant for statistical problems. The uniqueness condition is geometric and involves how the row span of the design matrix intersects the faces of the dual norm unit ball, which for SLOPE is given by the signed permutahedron. Further considerations based this condition also allow to derive results on sparsity and clustering features. In particular, we define the notion of a SLOPE pattern to describe both sparsity and clustering properties of this method and also provide a geometric characterization of accessible SLOPE patterns.