Landscape analysis of an improved power method for tensor decomposition

TL;DR

This paper analyzes the optimization landscape of the Subspace Power Method (SPM) for symmetric tensor decomposition, providing theoretical guarantees for its effectiveness in low-rank tensor recovery even in noisy settings.

Contribution

It offers a detailed landscape analysis of the SPM objective, deriving bounds that ensure convergence to tensor components and establishing near-global and global guarantees for low-rank tensor decomposition.

Findings

SPM objective has known maximum and exact rank-1 optima in low-rank cases.

Any second-order critical point with high objective value approximates a tensor component.

Numerical results favor SPM functional over traditional methods.

Abstract

In this work, we consider the optimization formulation for symmetric tensor decomposition recently introduced in the Subspace Power Method (SPM) of Kileel and Pereira. Unlike popular alternative functionals for tensor decomposition, the SPM objective function has the desirable properties that its maximal value is known in advance, and its global optima are exactly the rank-1 components of the tensor when the input is sufficiently low-rank. We analyze the non-convex optimization landscape associated with the SPM objective. Our analysis accounts for working with noisy tensors. We derive quantitative bounds such that any second-order critical point with SPM objective value exceeding the bound must equal a tensor component in the noiseless case, and must approximate a tensor component in the noisy case. For decomposing tensors of size $D^{\times m}$, we obtain a near-global guarantee up to rank $\widetilde{o}(D^{\lfloor m/2 \rfloor})$ under a random tensor model, and a global guarantee up to rank $\mathcal{O}(D)$ assuming deterministic frame conditions. This implies that SPM with suitable initialization is a provable, efficient, robust algorithm for low-rank symmetric tensor decomposition. We conclude with numerics that show a practical preferability for using the SPM functional over a more established counterpart.