Best rank-$k$ approximations for tensors: generalizing Eckart-Young

TL;DR

This paper generalizes the Eckart-Young theorem to tensors, showing that for a general tensor, the critical rank-at-most-$k$ tensors lie in a specific critical space, which is spanned by critical rank-one tensors.

Contribution

It extends the classical matrix Eckart-Young theorem to tensors, establishing that critical rank-at-most-$k$ tensors are contained in the critical space for general tensors.

Findings

Critical rank-at-most-$k$ tensors lie in the critical space $H_f$ for general tensors.

When the tensor format satisfies triangle inequalities, $H_f$ is spanned by critical rank-one tensors.

The tensor $f$ itself can be expressed as a linear combination of its critical rank-one tensors.

Abstract

Given a tensor $f$ in a Euclidean tensor space, we are interested in the critical points of the distance function from $f$ to the set of tensors of rank at most $k$, which we call the critical rank-at-most-$k$ tensors for $f$. When $f$ is a matrix, the critical rank-one matrices for $f$ correspond to the singular pairs of $f$. The critical rank-one tensors for $f$ lie in a linear subspace $H_f$, the critical space of $f$. Our main result is that, for any $k$, the critical rank-at-most-$k$ tensors for a sufficiently general $f$ also lie in the critical space $H_f$. This is the part of Eckart-Young Theorem that generalizes from matrices to tensors. Moreover, we show that when the tensor format satisfies the triangle inequalities, the critical space $H_f$ is spanned by the complex critical rank-one tensors. Since $f$ itself belongs to $H_f$, we deduce that also $f$ itself is a linear combination of its critical rank-one tensors.