Sparse Approximation by Semidefinite Programming

TL;DR

This paper introduces a novel approach to sparse approximation using semidefinite programming, offering a new perspective and algorithms with performance guarantees based on dictionary properties.

Contribution

It presents a semidefinite programming relaxation for sparse approximation and develops both randomized and derandomized algorithms with theoretical performance bounds.

Findings

Semidefinite relaxation provides a new framework for sparse approximation.

The randomized algorithm has performance guarantees related to coherence and restricted isometry.

Derandomization yields a deterministic algorithm with similar guarantees.

Abstract

The problem of sparse approximation and the closely related compressed sensing have received tremendous attention in the past decade. Primarily studied from the viewpoint of applied harmonic analysis and signal processing, there have been two dominant algorithmic approaches to this problem: Greedy methods called the matching pursuit (MP) and the linear programming based approaches called the basis pursuit (BP). The aim of the current paper is to bring a fresh perspective to sparse approximation by treating it as a combinatorial optimization problem and providing an algorithm based on the powerful optimization technique semidefinite programming (SDP). In particular, we show that there is a randomized algorithm based on a semidefinite relaxation of the problem with performance guarantees depending on the coherence and the restricted isometry constant of the dictionary used. We then show a derandomization of the algorithm based on the method of conditional probabilities.