Optimal variable selection and adaptive noisy Compressed Sensing

TL;DR

This paper introduces a polynomial-time, adaptive, and robust algorithm for support recovery in noisy compressed sensing, matching the performance of exhaustive search under standard Gaussian design matrices.

Contribution

It presents a nearly optimal, adaptive, and robust variable selection algorithm for noisy compressed sensing with theoretical guarantees.

Findings

Achieves exact support recovery under Gaussian design matrices.

Develops a polynomial-time, adaptive, and robust variable selection method.

Provides non-asymptotic minimax guarantees for the proposed algorithm.

Abstract

In the context of high-dimensional linear regression models, we propose an algorithm of exact support recovery in the setting of noisy compressed sensing where all entries of the design matrix are independent and identically distributed standard Gaussian. This algorithm achieves the same conditions of exact recovery as the exhaustive search (maximal likelihood) decoder, and has an advantage over the latter of being adaptive to all parameters of the problem and computable in polynomial time. The core of our analysis consists in the study of the non-asymptotic minimax Hamming risk of variable selection. This allows us to derive a procedure, which is nearly optimal in a non-asymptotic minimax sense. Then, we develop its adaptive version, and propose a robust variant of the method to handle datasets with outliers and heavy-tailed distributions of observations. The resulting polynomial time procedure is near optimal, adaptive to all parameters of the problem and also robust.