Information-Theoretic Guarantees for Recovering Low-Rank Tensors from Symmetric Rank-One Measurements

TL;DR

This paper establishes near-optimal sample complexity bounds for recovering low-rank symmetric tensors from rank-one measurements, with implications for neural network analysis, using advanced probabilistic and geometric tools.

Contribution

It introduces a theoretical framework with tight sample complexity bounds for tensor recovery from symmetric rank-one measurements, extending understanding in neural network contexts.

Findings

Near-optimal sample complexity bounds are proved.

The analysis employs Carbery-Wright inequality and orthogonal polynomials.

A lower bound based on Fano's inequality is provided.

Abstract

In this paper, we investigate the sample complexity of recovering tensors with low symmetric rank from symmetric rank-one measurements. This setting is particularly motivated by the study of higher-order interactions and the analysis of two-layer neural networks with polynomial activations (polynomial networks). Using a covering numbers argument, we analyze the performance of the symmetric rank minimization program and establish near-optimal sample complexity bounds when the underlying distribution is log-concave. Our measurement model involves random symmetric rank-one tensors, which lead to involved probability calculations. To address these challenges, we employ the Carbery-Wright inequality, a powerful tool for studying anti-concentration properties of random polynomials, and leverage orthogonal polynomials. Additionally, we provide a sample complexity lower bound based on Fano's inequality, and discuss broader implications of our results for two-layer polynomial networks.