Square Deal: Lower Bounds and Improved Relaxations for Tensor Recovery

TL;DR

This paper investigates the limitations of existing convex relaxations for low-rank tensor recovery, introduces a new relaxation that improves sample complexity bounds, and demonstrates its effectiveness through theoretical analysis and simulations.

Contribution

The authors propose a novel convex relaxation for tensor recovery that outperforms the traditional sum-of-nuclear-norms approach, reducing the number of measurements needed.

Findings

Sum-of-nuclear-norms approach requires (r n^{K-1}) observations

New relaxation achieves recovery with O(r^{\u2212floor{K/2}} n^{\u2212ceil{K/2}}) observations

Simulations indicate the new method outperforms traditional approaches in tensor completion

Abstract

Recovering a low-rank tensor from incomplete information is a recurring problem in signal processing and machine learning. The most popular convex relaxation of this problem minimizes the sum of the nuclear norms of the unfoldings of the tensor. We show that this approach can be substantially suboptimal: reliably recovering a $K$-way tensor of length $n$ and Tucker rank $r$ from Gaussian measurements requires $\Omega(r n^{K-1})$ observations. In contrast, a certain (intractable) nonconvex formulation needs only $O(r^K + nrK)$ observations. We introduce a very simple, new convex relaxation, which partially bridges this gap. Our new formulation succeeds with $O(r^{\lfloor K/2 \rfloor}n^{\lceil K/2 \rceil})$ observations. While these results pertain to Gaussian measurements, simulations strongly suggest that the new norm also outperforms the sum of nuclear norms for tensor completion from a random subset of entries. Our lower bound for the sum-of-nuclear-norms model follows from a new result on recovering signals with multiple sparse structures (e.g. sparse, low rank), which perhaps surprisingly demonstrates the significant suboptimality of the commonly used recovery approach via minimizing the sum of individual sparsity inducing norms (e.g. $l_1$, nuclear norm). Our new formulation for low-rank tensor recovery however opens the possibility in reducing the sample complexity by exploiting several structures jointly.