Variable selection, monotone likelihood ratio and group sparsity

TL;DR

This paper derives optimal variable selection methods with theoretical guarantees in sparse and group selection problems, providing explicit bounds and conditions for exact and near-complete recovery under various statistical models.

Contribution

It introduces a non-asymptotic minimax selector for sparse vectors, analyzes its computationally feasible approximation, and characterizes minimax risk under monotone likelihood ratio conditions.

Findings

The scan selector attains minimax expected Hamming risk within a factor of 2.

Explicit lower bounds are established under the monotone likelihood ratio property.

Conditions for exact and almost full recovery are derived for specific models.

Abstract

In the pivotal variable selection problem, we derive the exact non-asymptotic minimax selector over the class of all $s$-sparse vectors, which is also the Bayes selector with respect to the uniform prior. While this optimal selector is, in general, not realizable in polynomial time, we show that its tractable counterpart (the scan selector) attains the minimax expected Hamming risk to within factor 2, and is also exact minimax with respect to the probability of wrong recovery. As a consequence, we establish explicit lower bounds under the monotone likelihood ratio property and we obtain a tight characterization of the minimax risk in terms of the best separable selector risk. We apply these general results to derive necessary and sufficient conditions of exact and almost full recovery in the location model with light tail distributions and in the problem of group variable selection under Gaussian noise.