Guaranteed Non-Orthogonal Tensor Decomposition via Alternating Rank-$1$ Updates

TL;DR

This paper introduces an alternating rank-1 update algorithm for tensor decomposition with proven local and global convergence guarantees, capable of recovering overcomplete tensors even in noisy conditions.

Contribution

It provides the first convergence guarantees for an efficient tensor decomposition method applicable to overcomplete and asymmetric tensors, with simple initialization and perturbation analysis.

Findings

Guaranteed local convergence for third-order tensors with rank up to o(d^{1.5})

Global convergence under rank up to βd with simple initialization

Perturbation analysis for noisy tensor decomposition

Abstract

In this paper, we provide local and global convergence guarantees for recovering CP (Candecomp/Parafac) tensor decomposition. The main step of the proposed algorithm is a simple alternating rank-$1$ update which is the alternating version of the tensor power iteration adapted for asymmetric tensors. Local convergence guarantees are established for third order tensors of rank $k$ in $d$ dimensions, when $k=o \bigl( d^{1.5} \bigr)$ and the tensor components are incoherent. Thus, we can recover overcomplete tensor decomposition. We also strengthen the results to global convergence guarantees under stricter rank condition $k \le \beta d$ (for arbitrary constant $\beta > 1$) through a simple initialization procedure where the algorithm is initialized by top singular vectors of random tensor slices. Furthermore, the approximate local convergence guarantees for $p$-th order tensors are also provided under rank condition $k=o \bigl( d^{p/2} \bigr)$. The guarantees also include tight perturbation analysis given noisy tensor.