The Sparsity Gap: Uncertainty Principles Proportional to Dimension

TL;DR

This paper establishes an uncertainty principle for sparse signals in incoherent dictionaries, showing that nonoptimal representations require significantly more atoms unless they share many atoms with the sparsest representation.

Contribution

It introduces a new uncertainty principle indicating that alternative sparse representations are substantially less efficient unless they closely resemble the optimal one.

Findings

Nonoptimal representations need more atoms than the sparsest one.

Most signals with sparse representations have unique, highly privileged atoms.

The results extend to random sparse signals, reinforcing the uncertainty principle.

Abstract

In an incoherent dictionary, most signals that admit a sparse representation admit a unique sparse representation. In other words, there is no way to express the signal without using strictly more atoms. This work demonstrates that sparse signals typically enjoy a higher privilege: each nonoptimal representation of the signal requires far more atoms than the sparsest representation-unless it contains many of the same atoms as the sparsest representation. One impact of this finding is to confer a certain degree of legitimacy on the particular atoms that appear in a sparse representation. This result can also be viewed as an uncertainty principle for random sparse signals over an incoherent dictionary.