Joint k-step analysis of Orthogonal Matching Pursuit and Orthogonal Least Squares

TL;DR

This paper extends the analysis of Orthogonal Matching Pursuit (OMP) to Orthogonal Least Squares (OLS), providing conditions for exact support recovery and comparing their performance, especially on correlated dictionaries.

Contribution

It offers the first exact recovery analysis for OLS under the ERC and compares OMP and OLS in various dictionary scenarios.

Findings

OLS guarantees support recovery when ERC is met.

Neither OMP nor OLS is uniformly better; performance varies with dictionary correlation.

OLS may recover supports in fewer iterations than OMP for correlated dictionaries.

Abstract

Tropp's analysis of Orthogonal Matching Pursuit (OMP) using the Exact Recovery Condition (ERC) is extended to a first exact recovery analysis of Orthogonal Least Squares (OLS). We show that when the ERC is met, OLS is guaranteed to exactly recover the unknown support in at most k iterations. Moreover, we provide a closer look at the analysis of both OMP and OLS when the ERC is not fulfilled. The existence of dictionaries for which some subsets are never recovered by OMP is proved. This phenomenon also appears with basis pursuit where support recovery depends on the sign patterns, but it does not occur for OLS. Finally, numerical experiments show that none of the considered algorithms is uniformly better than the other but for correlated dictionaries, guaranteed exact recovery may be obtained after fewer iterations for OLS than for OMP.