Optimal rates for density and mode estimation with expand-and-sparsify representations

TL;DR

This paper investigates expand-and-sparsify representations, demonstrating their effectiveness for optimal density and mode estimation in high-dimensional statistical problems.

Contribution

It introduces the use of expand-and-sparsify representations for density and mode estimation, achieving minimax-optimal rates and providing simple algorithms for mode recovery.

Findings

Density estimator achieves minimax-optimal convergence rates.

Algorithms recover modes at near-optimal rates.

Representation captures sparse phenomena observed in sensory systems.

Abstract

Expand-and-sparsify representations are a class of theoretical models that capture sparse representation phenomena observed in the sensory systems of many animals. At a high level, these representations map an input $x \in \mathbb{R}^d$ to a much higher dimension $m \gg d$ via random linear projections before zeroing out all but the $k \ll m$ largest entries. The result is a $k$-sparse vector in $\{0,1\}^m$. We study the suitability of this representation for two fundamental statistical problems: density estimation and mode estimation. For density estimation, we show that a simple linear function of the expand-and-sparsify representation produces an estimator with minimax-optimal $\ell_{\infty}$ convergence rates. In mode estimation, we provide simple algorithms on top of our density estimator that recover single or multiple modes at optimal rates up to logarithmic factors under mild conditions.