A Geometric Perspective on the Injective Norm of Sums of Random Tensors

TL;DR

This paper develops new non-asymptotic concentration inequalities for the injective norm of random tensors, extending matrix inequalities to higher-order tensors with correlated entries, and applies geometric methods for proofs.

Contribution

It introduces tensor concentration inequalities based on covering numbers, providing nearly optimal bounds and a geometric proof of a weaker Non-Commutative Khintchine inequality.

Findings

Nearly optimal dimension-dependent bounds in certain regimes

Applications to tensor PCA and structured random tensors

Connections to coding theory lower bounds

Abstract

Matrix concentration inequalities, intimately connected to the Non-Commutative Khintchine inequality, have been an important tool in both applied and pure mathematics. We study tensor versions of these inequalities, and establish non-asymptotic inequalities for the $\ell_p$ injective norm of random tensors with correlated entries. In certain regimes of $p$ and the tensor order, our tensor concentration inequalities are nearly optimal in their dimension dependencies. We illustrate our result with applications to problems including structured models of random tensors and matrices, tensor PCA, and connections to lower bounds in coding theory. Our techniques are based on covering number estimates as opposed to operator theoretic tools, which also provide a geometric proof of a weaker version of the Non-Commutative Khintchine inequality, motivated by a question of Talagrand.