Adaptive variable selection in nonparametric sparse additive models

TL;DR

This paper develops an adaptive method for variable selection in high-dimensional nonparametric additive models, capable of nearly identifying most relevant components as the dimension grows.

Contribution

It introduces a new adaptive procedure for almost full variable selection in high-dimensional sparse additive models, achieving asymptotic minimax optimality.

Findings

Conditions for successful almost full variable selection are established.

The proposed procedure is adaptive to the sparsity parameter s.

The method performs optimally in the asymptotic minimax sense.

Abstract

We consider the problem of recovery of an unknown multivariate signal $f$ observed in a $d$-dimensional Gaussian white noise model of intensity $\varepsilon$. We assume that $f$ belongs to a class of smooth functions ${\cal F}^d\subset L_2([0,1]^d)$ and has an additive sparse structure determined by the parameter $s$, the number of non-zero univariate components contributing to $f$. We are interested in the case when $d=d_\varepsilon \to \infty$ as $\varepsilon \to 0$ and the parameter $s$ stays "small" relative to $d$. With these assumptions, the recovery problem in hand becomes that of determining which sparse additive components are non-zero. Attempting to reconstruct most non-zero components of $f$, but not all of them, we arrive at the problem of almost full variable selection in high-dimensional regression. For two different choices of ${\cal F}^d$, we establish conditions under which almost full variable selection is possible, and provide a procedure that gives almost full variable selection. The procedure does the best (in the asymptotically minimax sense) in selecting most non-zero components of $f$. Moreover, it is adaptive in the parameter $s$.