Sparse classification boundaries

TL;DR

This paper investigates the limits of sparse classification boundaries in high-dimensional settings with Gaussian and non-Gaussian noise, proposing classifiers that achieve these optimal boundaries.

Contribution

It derives the sharp classification boundary for sparse shifts in high dimensions and introduces classifiers that attain this boundary under various noise conditions.

Findings

Established the sharp classification boundary for Gaussian noise.

Proposed classifiers that achieve the optimal boundary.

Extended results to non-Gaussian noise satisfying the Cramér condition.

Abstract

Given a training sample of size $m$ from a $d$-dimensional population, we wish to allocate a new observation $Z\in \R^d$ to this population or to the noise. We suppose that the difference between the distribution of the population and that of the noise is only in a shift, which is a sparse vector. For the Gaussian noise, fixed sample size $m$, and the dimension $d$ that tends to infinity, we obtain the sharp classification boundary and we propose classifiers attaining this boundary. We also give extensions of this result to the case where the sample size $m$ depends on $d$ and satisfies the condition $(\log m)/\log d \to \gamma$, $0\le \gamma<1$, and to the case of non-Gaussian noise satisfying the Cram\'er condition.