Concentration of measure and generalized product of random vectors with an application to Hanson-Wright-like inequalities

TL;DR

This paper develops a new concentration inequality framework for functions of multiple random vectors, especially those resembling products, and applies it to generalized Hanson-Wright inequalities and random matrix analysis relevant to machine learning.

Contribution

It introduces a novel concentration of measure result for functionals with product-like variations, extending Hanson-Wright inequalities and analyzing random matrices in machine learning.

Findings

Generalized Hanson-Wright inequalities derived

Concentration results for random matrix $XDX^T$ and resolvent $Q$

Applications to statistical machine learning models

Abstract

Starting from concentration of measure hypotheses on $m$ random vectors $Z_1,\ldots, Z_m$, this article provides an expression of the concentration of functionals $\phi(Z_1,\ldots, Z_m)$ where the variations of $\phi$ on each variable depend on the product of the norms (or semi-norms) of the other variables (as if $\phi$ were a product). We illustrate the importance of this result through various generalizations of the Hanson-Wright concentration inequality as well as through a study of the random matrix $XDX^T$ and its resolvent $Q = (I_p - \frac{1}{n}XDX^T)^{-1}$, where $X$ and $D$ are random, which have fundamental interest in statistical machine learning applications.