Exact Recovery of Sparsely-Used Dictionaries

TL;DR

This paper proves that a small number of samples suffices to uniquely identify sparsely used dictionaries and introduces an efficient algorithm, ER-SpUD, that outperforms existing methods in recovering the true dictionary and coefficients.

Contribution

The paper provides a theoretical proof for sample complexity and develops ER-SpUD, a polynomial-time algorithm for exact dictionary recovery in sparse settings.

Findings

ER-SpUD recovers dictionaries with high probability

O(n log n) samples suffice for unique recovery

ER-SpUD outperforms state-of-the-art algorithms in simulations

Abstract

We consider the problem of learning sparsely used dictionaries with an arbitrary square dictionary and a random, sparse coefficient matrix. We prove that $O (n \log n)$ samples are sufficient to uniquely determine the coefficient matrix. Based on this proof, we design a polynomial-time algorithm, called Exact Recovery of Sparsely-Used Dictionaries (ER-SpUD), and prove that it probably recovers the dictionary and coefficient matrix when the coefficient matrix is sufficiently sparse. Simulation results show that ER-SpUD reveals the true dictionary as well as the coefficients with probability higher than many state-of-the-art algorithms.