A Riemannian Trust Region Method for the Canonical Tensor Rank Approximation Problem

TL;DR

This paper presents a novel Riemannian trust region method for low-rank tensor approximation, improving computational efficiency and robustness in solving the challenging canonical tensor rank approximation problem.

Contribution

It introduces a Riemannian Gauss-Newton approach with a new parametrization, a specialized retraction operator, and a hot restart mechanism for better convergence and speed.

Findings

Achieves up to three orders of magnitude faster solutions.

Effectively detects and escapes ill-conditioned tensor decompositions.

Outperforms existing state-of-the-art methods in numerical experiments.

Abstract

The canonical tensor rank approximation problem (TAP) consists of approximating a real-valued tensor by one of low canonical rank, which is a challenging non-linear, non-convex, constrained optimization problem, where the constraint set forms a non-smooth semi-algebraic set. We introduce a Riemannian Gauss-Newton method with trust region for solving small-scale, dense TAPs. The novelty of our approach is threefold. First, we parametrize the constraint set as the Cartesian product of Segre manifolds, hereby formulating the TAP as a Riemannian optimization problem, and we argue why this parametrization is among the theoretically best possible. Second, an original ST-HOSVD-based retraction operator is proposed. Third, we introduce a hot restart mechanism that efficiently detects when the optimization process is tending to an ill-conditioned tensor rank decomposition and which often yields a quick escape path from such spurious decompositions. Numerical experiments show improvements of up to three orders of magnitude in terms of the expected time to compute a successful solution over existing state-of-the-art methods.