Subtracting a best rank-1 approximation may increase tensor rank

TL;DR

This paper proves that subtracting a best rank-1 approximation from a generic 2x2x2 tensor often increases its rank, highlighting limitations in tensor approximation methods.

Contribution

It provides a mathematical analysis showing that subtracting a best rank-1 approximation can increase tensor rank for generic 2x2x2 tensors, which was previously observed numerically.

Findings

Subtracting a best rank-1 approximation often increases tensor rank.

For generic 2x2x2 tensors, the resulting tensor after subtraction lies on the boundary of rank sets.

This phenomenon occurs regardless of tensor symmetry.

Abstract

It has been shown that a best rank-R approximation of an order-k tensor may not exist when R>1 and k>2. This poses a serious problem to data analysts using tensor decompositions. It has been observed numerically that, generally, this issue cannot be solved by consecutively computing and subtracting best rank-1 approximations. The reason for this is that subtracting a best rank-1 approximation generally does not decrease tensor rank. In this paper, we provide a mathematical treatment of this property for real-valued 2x2x2 tensors, with symmetric tensors as a special case. Regardless of the symmetry, we show that for generic 2x2x2 tensors (which have rank 2 or 3), subtracting a best rank-1 approximation results in a tensor that has rank 3 and lies on the boundary between the rank-2 and rank-3 sets. Hence, for a typical tensor of rank 2, subtracting a best rank-1 approximation increases the tensor rank.