Average performance of the sparsest approximation using a general dictionary

TL;DR

This paper analyzes the probability and expected number of non-zero coefficients in the sparsest approximation of data using arbitrary dictionaries and norms, providing bounds and precise descriptions of data sets leading to K-sparse solutions.

Contribution

It offers a detailed characterization of data sets that produce K-sparse solutions and derives bounds on their measure and probability, extending understanding of sparsity in general dictionary settings.

Findings

Derived bounds on the Lebesgue measure of data sets yielding K-sparse solutions

Provided probability estimates for obtaining K-sparse representations

Calculated the expected number of non-zero components in the approximation

Abstract

We consider the minimization of the number of non-zero coefficients (the $\ell_0$ "norm") of the representation of a data set in terms of a dictionary under a fidelity constraint. (Both the dictionary and the norm defining the constraint are arbitrary.) This (nonconvex) optimization problem naturally leads to the sparsest representations, compared with other functionals instead of the $\ell_0$ "norm". Our goal is to measure the sets of data yielding a $K$-sparse solution--i.e. involving $K$ non-zero components. Data are assumed uniformly distributed on a domain defined by any norm--to be chosen by the user. A precise description of these sets of data is given and relevant bounds on the Lebesgue measure of these sets are derived. They naturally lead to bound the probability of getting a $K$-sparse solution. We also express the expectation of the number of non-zero components. We further specify these results in the case of the Euclidean norm, the dictionary being arbitrary.