Tensor Ring Decomposition: Optimization Landscape and One-loop Convergence of Alternating Least Squares

TL;DR

This paper analyzes the optimization landscape of tensor ring decomposition, revealing a transition from local minima to one-loop convergence of ALS as bond dimension increases, supported by theoretical and numerical evidence.

Contribution

It provides a detailed theoretical analysis of the optimization landscape and convergence behavior of ALS in tensor ring decomposition, highlighting a phase transition based on bond dimension.

Findings

Existence of spurious local minima at certain bond dimensions.

One-loop convergence of ALS when bond dimension is sufficiently large.

Numerical experiments confirming theoretical results.

Abstract

In this work, we study the tensor ring decomposition and its associated numerical algorithms. We establish a sharp transition of algorithmic difficulty of the optimization problem as the bond dimension increases: On one hand, we show the existence of spurious local minima for the optimization landscape even when the tensor ring format is much over-parameterized, i.e., with bond dimension much larger than that of the true target tensor. On the other hand, when the bond dimension is further increased, we establish one-loop convergence for alternating least square algorithm for tensor ring decomposition. The theoretical results are complemented by numerical experiments for both local minimum and one-loop convergence for the alternating least square algorithm.