A measure concentration effect for matrices of high, higher, and even higher dimension

TL;DR

This paper investigates how, in high-dimensional spaces, the set of vectors for which a matrix acts nearly isometrically becomes overwhelmingly large, revealing a concentration phenomenon related to random projections.

Contribution

It establishes a measure concentration effect for high-dimensional matrices and provides exact probabilities related to the random projection theorem.

Findings

Sets of vectors with near-isometric behavior fill almost the entire space in high dimensions.

The measure concentration depends on extremal singular values and dimension ratios.

Exact probabilities for the random projection theorem are derived.

Abstract

Let $n>m$, and let $A$ be an $(m\times n)$-matrix of full rank. Then obviously the estimate $\|Ax\|\leq\|A\|\|x\|$ holds for the euclidean norm of $x$ and $Ax$ and the spectral norm as the assigned matrix norm. We study the sets of all $x$ for which, for fixed $\delta<1$, conversely $\|Ax\|\geq\delta\,\|A\|\|x\|$ holds. It turns out that these sets fill, in the high-dimensional case, almost the complete space once $\delta$ falls below a bound that depends on the extremal singular values of $A$ and on the ratio of the dimensions. This effect has much to do with the random projection theorem, which plays an important role in the data sciences. As a byproduct, we calculate the probabilities this theorem deals with exactly.