Sample Complexity Bounds for Learning High-dimensional Simplices in Noisy Regimes

TL;DR

This paper establishes sample complexity bounds for learning high-dimensional simplices from noisy data, demonstrating that under certain SNR conditions, the complexity matches the noiseless case, and introduces a Fourier-based recovery technique.

Contribution

It provides the first theoretical bounds for learning simplices in noisy regimes and introduces a novel Fourier analysis method for distribution recovery.

Findings

Sample complexity bound: $n \,\ge\, (K^2/\varepsilon^2) e^{\Omega(K/\mathrm{SNR}^2)}$

As long as SNR ≥ Ω(√K), noisy and noiseless sample complexities are comparable

Proposes a Fourier-based technique for recovering distributions from Gaussian noise

Abstract

In this paper, we find a sample complexity bound for learning a simplex from noisy samples. Assume a dataset of size $n$ is given which includes i.i.d. samples drawn from a uniform distribution over an unknown simplex in $\mathbb{R}^K$, where samples are assumed to be corrupted by a multi-variate additive Gaussian noise of an arbitrary magnitude. We prove the existence of an algorithm that with high probability outputs a simplex having a $\ell_2$ distance of at most $\varepsilon$ from the true simplex (for any $\varepsilon>0$). Also, we theoretically show that in order to achieve this bound, it is sufficient to have $n\ge\left(K^2/\varepsilon^2\right)e^{\Omega\left(K/\mathrm{SNR}^2\right)}$ samples, where $\mathrm{SNR}$ stands for the signal-to-noise ratio. This result solves an important open problem and shows as long as $\mathrm{SNR}\ge\Omega\left(K^{1/2}\right)$, the sample complexity of the noisy regime has the same order to that of the noiseless case. Our proofs are a combination of the so-called sample compression technique in \citep{ashtiani2018nearly}, mathematical tools from high-dimensional geometry, and Fourier analysis. In particular, we have proposed a general Fourier-based technique for recovery of a more general class of distribution families from additive Gaussian noise, which can be further used in a variety of other related problems.