Refined Least Squares for Support Recovery

TL;DR

This paper introduces Refined Least Squares (RLS), a novel method for exact support recovery in noisy linear models that outperforms existing algorithms by averaging multiple least squares solutions.

Contribution

The paper presents RLS, a new support recovery algorithm that leverages averaging multiple least squares solutions for improved accuracy in noisy settings.

Findings

RLS outperforms state-of-the-art algorithms in various settings.

The method effectively recovers the support of sparse vectors with Gaussian measurement matrices.

Support is identified by selecting the most significant coefficients of the averaged solution.

Abstract

We study the problem of exact support recovery based on noisy observations and present Refined Least Squares (RLS). Given a set of noisy measurement $$ \myvec{y} = \myvec{X}\myvec{\theta}^* + \myvec{\omega},$$ and $\myvec{X} \in \mathbb{R}^{N \times D}$ which is a (known) Gaussian matrix and $\myvec{\omega} \in \mathbb{R}^N$ is an (unknown) Gaussian noise vector, our goal is to recover the support of the (unknown) sparse vector $\myvec{\theta}^* \in \left\{-1,0,1\right\}^D$. To recover the support of the $\myvec{\theta}^*$ we use an average of multiple least squares solutions, each computed based on a subset of the full set of equations. The support is estimated by identifying the most significant coefficients of the average least squares solution. We demonstrate that in a wide variety of settings our method outperforms state-of-the-art support recovery algorithms.