Structured Sparsity: Discrete and Convex approaches

TL;DR

This paper explores structured sparsity models in compressive sensing, analyzing discrete and convex approaches, and demonstrating their advantages and applications in signal recovery and interpretability.

Contribution

It provides a comprehensive analysis of discrete and convex structured sparsity models, including group, dispersive, and hierarchical models, with solutions and applications.

Findings

Convex relaxations effectively approximate discrete structured sparsity models.

Structured sparsity models improve signal recovery and interpretability.

Efficient optimization methods enable practical applications of structured sparsity.

Abstract

Compressive sensing (CS) exploits sparsity to recover sparse or compressible signals from dimensionality reducing, non-adaptive sensing mechanisms. Sparsity is also used to enhance interpretability in machine learning and statistics applications: While the ambient dimension is vast in modern data analysis problems, the relevant information therein typically resides in a much lower dimensional space. However, many solutions proposed nowadays do not leverage the true underlying structure. Recent results in CS extend the simple sparsity idea to more sophisticated {\em structured} sparsity models, which describe the interdependency between the nonzero components of a signal, allowing to increase the interpretability of the results and lead to better recovery performance. In order to better understand the impact of structured sparsity, in this chapter we analyze the connections between the discrete models and their convex relaxations, highlighting their relative advantages. We start with the general group sparse model and then elaborate on two important special cases: the dispersive and the hierarchical models. For each, we present the models in their discrete nature, discuss how to solve the ensuing discrete problems and then describe convex relaxations. We also consider more general structures as defined by set functions and present their convex proxies. Further, we discuss efficient optimization solutions for structured sparsity problems and illustrate structured sparsity in action via three applications.