Randomized Alternating Least Squares for Canonical Tensor Decompositions: Application to a PDE with Random Data

TL;DR

This paper proposes a randomized ALS algorithm for tensor rank reduction that improves numerical stability and accuracy, demonstrated through examples including a PDE with random data.

Contribution

It introduces a probabilistic, randomized approach to ALS that reduces condition numbers and enhances solution accuracy for tensor decompositions.

Findings

Reduces condition numbers of ALS matrices

Maintains comparable accuracy to standard ALS

Improves PDE solution accuracy with random data

Abstract

This paper introduces a randomized variation of the alternating least squares (ALS) algorithm for rank reduction of canonical tensor formats. The aim is to address the potential numerical ill-conditioning of least squares matrices at each ALS iteration. The proposed algorithm, dubbed randomized ALS, mitigates large condition numbers via projections onto random tensors, a technique inspired by well-established randomized projection methods for solving overdetermined least squares problems in a matrix setting. A probabilistic bound on the condition numbers of the randomized ALS matrices is provided, demonstrating reductions relative to their standard counterparts. Additionally, results are provided that guarantee comparable accuracy of the randomized ALS solution at each iteration. The performance of the randomized algorithm is studied with three examples, including manufactured tensors and an elliptic PDE with random inputs. In particular, for the latter, tests illustrate not only improvements in condition numbers, but also improved accuracy of the iterative solver for the PDE solution represented in a canonical tensor format.