Variable selection with Hamming loss

TL;DR

This paper establishes non-asymptotic bounds and explicit minimax selectors for variable selection under Hamming loss in Gaussian models, extending results to dependent, non-Gaussian data, and crowdsourcing, with adaptive procedures for recovery.

Contribution

It provides the first explicit minimax risk bounds and selectors for variable selection under Hamming loss, including extensions to dependent and non-Gaussian data, with adaptive methods for recovery.

Findings

Derived non-asymptotic minimax risk bounds

Explicit minimax selectors for variable selection

Adaptive procedures for near-perfect recovery

Abstract

We derive non-asymptotic bounds for the minimax risk of variable selection under expected Hamming loss in the Gaussian mean model in $\mathbb{R}^d$ for classes of $s$-sparse vectors separated from 0 by a constant $a > 0$. In some cases, we get exact expressions for the nonasymptotic minimax risk as a function of $d, s, a$ and find explicitly the minimax selectors. These results are extended to dependent or non-Gaussian observations and to the problem of crowdsourcing. Analogous conclusions are obtained for the probability of wrong recovery of the sparsity pattern. As corollaries, we derive necessary and sufficient conditions for such asymptotic properties as almost full recovery and exact recovery. Moreover, we propose data-driven selectors that provide almost full and exact recovery adaptively to the parameters of the classes.