On the average condition number of tensor rank decompositions

TL;DR

This paper analyzes the expected sensitivity of tensor rank decompositions, revealing that certain low-dimensional tensors are highly sensitive to perturbations, while higher-dimensional tensors tend to be more stable.

Contribution

It provides the first theoretical and empirical analysis of the average condition number for tensor decompositions, highlighting differences based on tensor size and rank.

Findings

Expected condition number is infinite for certain $n_1\times n_2 \times 2$ tensors.

Tensors with all dimensions at least 3 have finite average condition numbers.

The results suggest a gap in sensitivity between low-dimensional and higher-dimensional tensors.

Abstract

We compute the expected value of powers of the geometric condition number of random tensor rank decompositions. It is shown in particular that the expected value of the condition number of $n_1\times n_2 \times 2$ tensors with a random rank-$r$ decomposition, given by factor matrices with independent and identically distributed standard normal entries, is infinite. This entails that it is expected and probable that such a rank-$r$ decomposition is sensitive to perturbations of the tensor. Moreover, it provides concrete further evidence that tensor decomposition can be a challenging problem, also from the numerical point of view. On the other hand, we provide strong theoretical and empirical evidence that tensors of size $n_1~\times~n_2~\times~n_3$ with all $n_1,n_2,n_3 \ge 3$ have a finite average condition number. This suggests there exists a gap in the expected sensitivity of tensors between those of format $n_1\times n_2 \times 2$ and other order-3 tensors. For establishing these results, we show that a natural weighted distance from a tensor rank decomposition to the locus of ill-posed decompositions with an infinite geometric condition number is bounded from below by the inverse of this condition number. That is, we prove one inequality towards a so-called condition number theorem for the tensor rank decomposition.