Order statistics of vectors with dependent coordinates, and the Karhunen-Lo\`eve basis

TL;DR

This paper investigates the order statistics of Gaussian vectors with dependent coordinates, demonstrating the near-optimality of the Karhunen-Loève basis for nonlinear signal approximation and providing new insights into the behavior of order statistics in random vectors.

Contribution

It proves a universal bound relating order statistics of Gaussian vectors under orthogonal transformations, addressing an open question about the optimality of the Karhunen-Loève basis.

Findings

Establishes a universal inequality for order statistics of Gaussian vectors.

Shows the near-optimality of the Karhunen-Loève basis for nonlinear approximation.

Provides new relations for order statistics of general random vectors.

Abstract

Let $X$ be an $n$-dimensional random centered Gaussian vector with independent but not identically distributed coordinates and let $T$ be an orthogonal trasformation of $\mathbb R^n$. We show that the random vector $Y=T(X)$ satisfies $$\mathbb E\sum\limits_{j=1}^k j\mbox{-}\min_{i\leq n}{X_{i}}^2 \leq C\mathbb E\sum\limits_{j=1}^k j\mbox{-}\min_{i\leq n}{Y_{i}}^2$$ for all $k<n$, where "$j\mbox{-}\min$" denotes the $j$-th smallest component of corresponding vector and $C>0$ is a universal constant. This resolves (up to a multiplicative constant) an old question of S.Mallat and O.Zeitouni regarding optimality of the Karhunen-Loeve basis for the nonlinear signal approximation. As a by-product we obtain some relations for order statistics of random vectors (not only Gaussian) which are of independent interest.