The Epsilon-Alternating Least Squares for Orthogonal Low-Rank Tensor Approximation and Its Global Convergence

TL;DR

This paper introduces the epsilon-ALS algorithm for orthogonal low-rank tensor approximation, proving its global convergence and proposing an initialization method, with numerical results demonstrating its efficiency and effectiveness.

Contribution

The paper establishes the global convergence of epsilon-ALS for orthogonal tensor approximation and introduces a provably effective initialization procedure.

Findings

Epsilon-ALS globally converges to a KKT point for all tensors.

The proposed initialization method has a provable lower bound.

Numerical experiments show promising efficiency and effectiveness.

Abstract

The epsilon alternating least squares ($\epsilon$-ALS) is developed and analyzed for canonical polyadic decomposition (approximation) of a higher-order tensor where one or more of the factor matrices are assumed to be columnwisely orthonormal. It is shown that the algorithm globally converges to a KKT point for all tensors without any assumption. For the original ALS, by further studying the properties of the polar decomposition, we also establish its global convergence under a reality assumption not stronger than those in the literature. These results completely address a question concerning the global convergence raised in [L. Wang, M. T. Chu and B. Yu, \emph{SIAM J. Matrix Anal. Appl.}, 36 (2015), pp. 1--19]. In addition, an initialization procedure is proposed, which possesses a provable lower bound when the number of columnwisely orthonormal factors is one. Armed with this initialization procedure, numerical experiments show that the $\epsilon$-ALS exhibits a promising performance in terms of efficiency and effectiveness.