Linear Convergence of an Alternating Polar Decomposition Method for Low Rank Orthogonal Tensor Approximations

TL;DR

This paper introduces an improved alternating polar decomposition algorithm for low rank orthogonal tensor approximation, proving its global convergence, sublinear rate, and linear convergence under broad conditions.

Contribution

The paper presents an enhanced algorithm for LROTA with rigorous convergence analysis, including global convergence, explicit sublinear rate, and R-linear convergence for generic tensors.

Findings

The algorithm converges globally to a KKT point.

It has a sharper sublinear convergence rate than typical first-order methods.

It achieves R-linear convergence for generic tensors without additional assumptions.

Abstract

Low rank orthogonal tensor approximation (LROTA) is an important problem in tensor computations and their applications. A classical and widely used algorithm is the alternating polar decomposition method (APD). In this article, an improved version iAPD of the classical APD is proposed. For the first time, all the following four fundamental properties are established for iAPD: (i) the algorithm converges globally and the whole sequence converges to a KKT point without any assumption; (ii) it exhibits an overall sublinear convergence with an explicit rate which is sharper than the usual $O(1/k)$ for first order methods in optimization; (iii) more importantly, it converges $R$-linearly for a generic tensor without any assumption; (iv) for almost all LROTA problems, iAPD reduces to APD after finitely many iterations if it converges to a local minimizer.