Separating Oblivious and Adaptive Models of Variable Selection

TL;DR

This paper explores the differences between oblivious and adaptive models in sparse recovery with $\, ext{ extonehalfspace}$ error guarantees, revealing a significant sample complexity gap and introducing a partially-adaptive model with promising results.

Contribution

It establishes a provable separation between oblivious and adaptive models in $\, ext{ extonehalfspace}$ sparse recovery, highlighting different sample complexities and introducing a partially-adaptive approach.

Findings

Oblivious model achieves near-linear time recovery with $\,k\, ext{log}\,d$ samples.

Adaptive model requires at least $\,k^2$ samples for similar guarantees.

Partially-adaptive model can attain variable selection with $\,k\, ext{log}\,d$ measurements.

Abstract

Sparse recovery is among the most well-studied problems in learning theory and high-dimensional statistics. In this work, we investigate the statistical and computational landscapes of sparse recovery with $\ell_\infty$ error guarantees. This variant of the problem is motivated by \emph{variable selection} tasks, where the goal is to estimate the support of a $k$-sparse signal in $\mathbb{R}^d$. Our main contribution is a provable separation between the \emph{oblivious} (``for each'') and \emph{adaptive} (``for all'') models of $\ell_\infty$ sparse recovery. We show that under an oblivious model, the optimal $\ell_\infty$ error is attainable in near-linear time with $\approx k\log d$ samples, whereas in an adaptive model, $\gtrsim k^2$ samples are necessary for any algorithm to achieve this bound. This establishes a surprising contrast with the standard $\ell_2$ setting, where $\approx k \log d$ samples suffice even for adaptive sparse recovery. We conclude with a preliminary examination of a \emph{partially-adaptive} model, where we show nontrivial variable selection guarantees are possible with $\approx k\log d$ measurements.