Fast algorithm for overcomplete order-3 tensor decomposition

TL;DR

This paper introduces a fast spectral algorithm for decomposing overcomplete third-order tensors of high rank, significantly improving efficiency over previous methods and relying on simple linear algebra operations.

Contribution

It presents the first fast spectral algorithm for overcomplete third-order tensor decomposition with rank up to nearly $d^{3/2}$, using tensor network techniques and matrix multiplications.

Findings

Achieves tensor decomposition in $O(d^{6.05})$ time.

Handles tensors with rank up to $O(d^{3/2}/ ext{polylog}(d))$.

Outperforms prior sum-of-squares based algorithms in speed and rank capability.

Abstract

We develop the first fast spectral algorithm to decompose a random third-order tensor over $\mathbb{R}^d$ of rank up to $O(d^{3/2}/\text{polylog}(d))$. Our algorithm only involves simple linear algebra operations and can recover all components in time $O(d^{6.05})$ under the current matrix multiplication time. Prior to this work, comparable guarantees could only be achieved via sum-of-squares [Ma, Shi, Steurer 2016]. In contrast, fast algorithms [Hopkins, Schramm, Shi, Steurer 2016] could only decompose tensors of rank at most $O(d^{4/3}/\text{polylog}(d))$. Our algorithmic result rests on two key ingredients. A clean lifting of the third-order tensor to a sixth-order tensor, which can be expressed in the language of tensor networks. A careful decomposition of the tensor network into a sequence of rectangular matrix multiplications, which allows us to have a fast implementation of the algorithm.