Group-Sparse Model Selection: Hardness and Relaxations

TL;DR

This paper explores the computational complexity of group-sparse model selection, establishing its NP-hardness, and proposes tractable relaxations and algorithms for specific group structures, including hierarchical sparsity.

Contribution

It introduces a combinatorial framework for group-model selection, identifies tractable cases, and extends models to hierarchical sparsity with analysis of trade-offs.

Findings

Group-model selection is NP-hard, equivalent to weighted maximum coverage.

Certain group structures allow polynomial-time model selection via dynamic programming.

Convex relaxations are tractable for totally unimodular constraints.

Abstract

Group-based sparsity models are proven instrumental in linear regression problems for recovering signals from much fewer measurements than standard compressive sensing. The main promise of these models is the recovery of "interpretable" signals through the identification of their constituent groups. In this paper, we establish a combinatorial framework for group-model selection problems and highlight the underlying tractability issues. In particular, we show that the group-model selection problem is equivalent to the well-known NP-hard weighted maximum coverage problem (WMC). Leveraging a graph-based understanding of group models, we describe group structures which enable correct model selection in polynomial time via dynamic programming. Furthermore, group structures that lead to totally unimodular constraints have tractable discrete as well as convex relaxations. We also present a generalization of the group-model that allows for within group sparsity, which can be used to model hierarchical sparsity. Finally, we study the Pareto frontier of group-sparse approximations for two tractable models, among which the tree sparsity model, and illustrate selection and computation trade-offs between our framework and the existing convex relaxations.