Spectral norm of random tensors

TL;DR

This paper establishes a bound on the spectral norm of random tensors, showing it scales with the square root of the sum of dimensions times a logarithmic factor, with implications for tensor norm relaxations.

Contribution

It provides a new spectral norm bound for random tensors under sub-Gaussian assumptions, improving understanding of tensor convex relaxations and their sample complexity.

Findings

Spectral norm scales as O(√(sum of dimensions) log(K))

Convex relaxation based on spectral norm has linear sample complexity

Bounds apply to tensors with sub-Gaussian entries

Abstract

We show that the spectral norm of a random $n_1\times n_2\times \cdots \times n_K$ tensor (or higher-order array) scales as $O\left(\sqrt{(\sum_{k=1}^{K}n_k)\log(K)}\right)$ under some sub-Gaussian assumption on the entries. The proof is based on a covering number argument. Since the spectral norm is dual to the tensor nuclear norm (the tightest convex relaxation of the set of rank one tensors), the bound implies that the convex relaxation yields sample complexity that is linear in (the sum of) the number of dimensions, which is much smaller than other recently proposed convex relaxations of tensor rank that use unfolding.