A product-CLT and its application in invariance principle of random projection

TL;DR

This paper investigates how random projections preserve the distribution of inner products between independent random vectors, establishing a probabilistic invariance principle via a novel product-CLT with convergence rates.

Contribution

It introduces a new product-CLT for dependent sequences and applies it to prove the invariance of inner product distributions under random projections.

Findings

Inner product distribution is preserved within an error bound of O(1/√n + 1/√m)

Established a rate of convergence similar to Berry-Esseen theorem for the product-CLT

Provided theoretical foundation for probabilistic properties under Johnson-Lindenstrauss type embeddings

Abstract

Johnson-Lindenstrauss lemma states random projections can be used as a topology preserving embedding technique for fixed vectors. In this paper, we try to understand how random projections affect probabilistic properties of random vectors. In particular we prove the distribution of inner product of two independent random vectors $X, Z \in {R}^n$ is preserved by random projection $S:{R}^n \to {R}^m$. More precisely, \[ \sup_t \left| \text{P}(\frac{1}{C_{m,n}} X^TS^TSZ <t) - \text{P}(\frac{1}{\sqrt{n}} X^TZ<t) \right| \le O\left(\frac{1}{\sqrt{n}}+ \frac{1}{\sqrt{m}} \right) \] This is achieved by proving a general central limit theorem (product-CLT) for $\sum_{k=1}^{n} X_k Y_k$, where $\{X_k\}$ is a martingale difference sequence, and $\{Y_k\}$ has dependency within the sequence. We also obtain the rate of convergence in the spirit of Berry-Esseen theorem.