Rank-Extreme Association of Gaussian Vectors and Low-Rank Detection

TL;DR

This paper establishes universal bounds and limiting distributions for maximal inner products and norms in high-dimensional Gaussian and elliptically distributed data, enabling efficient low-rank detection without spectral analysis.

Contribution

It introduces new asymptotic bounds and distributional results for inner products and norms, facilitating low-rank detection in high-dimensional Gaussian data.

Findings

Universal bounds on maximal inner products and norms.

Limiting distributions for extreme inner products and norms.

A fast low-rank detection method based on these theoretical results.

Abstract

We study the spherical cap packing problem with a probabilistic approach. Such probabilistic considerations result in an asymptotic sharp universal uniform bound on the maximal inner product between any set of unit vectors and a stochastically independent uniformly distributed unit vector. When the set of unit vectors are themselves independently uniformly distributed, we further develop the extreme value distribution limit of the maximal inner product, which characterizes its uncertainty around the bound. As applications of the above asymptotic results, we derive (1) an asymptotic sharp universal uniform bound on the maximal spurious correlation, as well as its uniform convergence in distribution when the explanatory variables are independently Gaussian distributed; and (2) an asymptotic sharp universal bound on the maximum norm of a low-rank elliptically distributed vector, as well as related limiting distributions. With these results, we develop a fast detection method for a low-rank structure in high-dimensional Gaussian data without using the spectrum information.