Guaranteed Simultaneous Asymmetric Tensor Decomposition via Orthogonalized Alternating Least Squares

TL;DR

This paper introduces a guaranteed method for simultaneous orthogonal tensor decomposition that works almost surely in noiseless cases and is robust to small noise, improving theoretical understanding and practical reliability.

Contribution

It provides the first theoretical guarantees for simultaneous orthogonal asymmetric tensor decomposition with a novel Slice-Based Initialization and tensor subspace iteration approach.

Findings

Guarantees recovery of top r components almost surely in noiseless cases.

Achieves high-probability recovery under bounded noise.

Introduces a novel Slice-Based Initialization method.

Abstract

Tensor CANDECOMP/PARAFAC (CP) decomposition is an important tool that solves a wide class of machine learning problems. Existing popular approaches recover components one by one, not necessarily in the order of larger components first. Recently developed simultaneous power method obtains only a high probability recovery of top $r$ components even when the observed tensor is noiseless. We propose a Slicing Initialized Alternating Subspace Iteration (s-ASI) method that is guaranteed to recover top $r$ components ($\epsilon$-close) simultaneously for (a)symmetric tensors almost surely under the noiseless case (with high probability for a bounded noise) using $O(\log(\log \frac{1}{\epsilon}))$ steps of tensor subspace iterations. Our s-ASI method introduces a Slice-Based Initialization that runs $O(1/\log(\frac{\lambda_r}{\lambda_{r+1}}))$ steps of matrix subspace iterations, where $\lambda_r$ denotes the r-th top singular value of the tensor. We are the first to provide a theoretical guarantee on simultaneous orthogonal asymmetric tensor decomposition. Under the noiseless case, we are the first to provide an \emph{almost sure} theoretical guarantee on simultaneous orthogonal tensor decomposition. When tensor is noisy, our algorithm for asymmetric tensor is robust to noise smaller than $\min\{O(\frac{(\lambda_r - \lambda_{r+1})\epsilon}{\sqrt{r}}), O(\delta_0\frac{\lambda_r -\lambda_{r+1}}{\sqrt{d}})\}$, where $\delta_0$ is a small constant proportional to the probability of bad initializations in the noisy setting.