Pencil-based algorithms for tensor rank decomposition are not stable

TL;DR

This paper demonstrates that popular tensor rank decomposition algorithms based on pencil reduction are numerically unstable for an open set of tensors, especially when the third dimension is small, leading to potential loss of precision.

Contribution

The paper proves the instability of pencil-based tensor decomposition algorithms for certain tensors and provides bounds on the condition number distribution for random tensors.

Findings

Algorithms can be arbitrarily unstable on an open set of tensors.

Condition number can be much larger for tensors with small third dimension.

Random tensor decompositions may lose several digits of precision.

Abstract

We prove the existence of an open set of $n_1\times n_2 \times n_3$ tensors of rank $r$ on which a popular and efficient class of algorithms for computing tensor rank decompositions based on a reduction to a linear matrix pencil, typically followed by a generalized eigendecomposition, is arbitrarily numerically forward unstable. Our analysis shows that this problem is caused by the fact that the condition number of the tensor rank decomposition can be much larger for $n_1 \times n_2 \times 2$ tensors than for the $n_1\times n_2 \times n_3$ input tensor. Moreover, we present a lower bound for the limiting distribution of the condition number of random tensor rank decompositions of third-order tensors. The numerical experiments illustrate that for random tensor rank decompositions one should anticipate a loss of precision of a few digits.