Sparse Approximation Over the Cube

TL;DR

This paper analyzes a complex NP-hard sparse approximation problem over the cube, exploring relaxations and providing probabilistic and deterministic bounds, leading to an efficient algorithm for certain matrix classes.

Contribution

It introduces a relaxation of the sparse approximation problem, analyzes its exactness probabilistically, and offers bounds and an algorithm for specific matrix types.

Findings

Relaxation can be exact under certain probabilistic conditions.

Provides bounds on the distance between original and relaxed solutions.

Develops a polynomial-time algorithm for fixed matrix parameters.

Abstract

This paper presents an anlysis of the NP-hard minimization problem $\min \{\|b - Ax\|_2: \ x \in [0,1]^n, | \text{supp}(x) | \leq \sigma\}$, where $\text{supp}(x) = \{i \in [n]: x_i \neq 0\}$ and $\sigma$ is a positive integer. The object of investigation is a natural relaxation where we replace $| \text{supp}(x) | \leq \sigma$ by $\sum_i x_i \leq \sigma$. Our analysis includes a probabilistic view on when the relaxation is exact. We also consider the problem from a deterministic point of view and provide a bound on the distance between the images of optimal solutions of the original problem and its relaxation under $A$. This leads to an algorithm for generic matrices $A \in \mathbb{Z}^{m \times n}$ and achieves a polynomial running time provided that $m$ and $\|A\|_{\infty}$ are fixed.