Best Rank-One Tensor Approximation and Parallel Update Algorithm for CPD

TL;DR

This paper introduces a parallel update algorithm for tensor decomposition that exploits best rank-1 tensor approximation, utilizing all-at-once methods based on Levenberg-Marquardt and rotational updates, improving efficiency for complex tensors.

Contribution

It develops new all-at-once algorithms for best rank-1 tensor approximation, enabling parallel updates in CPD, with closed-form solutions for small tensors and an ALS approach for higher orders.

Findings

The LM algorithm has comparable complexity to first-order methods.

The rotational method simplifies to solving small 2x2x2 tensor approximations.

The algorithm effectively decomposes complex tensors related to matrix multiplication.

Abstract

A novel algorithm is proposed for CANDECOMP/PARAFAC tensor decomposition to exploit best rank-1 tensor approximation. Different from the existing algorithms, our algorithm updates rank-1 tensors simultaneously in parallel. In order to achieve this, we develop new all-at-once algorithms for best rank-1 tensor approximation based on the Levenberg-Marquardt method and the rotational update. We show that the LM algorithm has the same complexity of first-order optimisation algorithms, while the rotational method leads to solving the best rank-1 approximation of tensors of size $2 \times 2 \times \cdots \times 2$. We derive a closed-form expression of the best rank-1 tensor of $2\times 2 \times 2$ tensors and present an ALS algorithm which updates 3 component at a time for higher order tensors. The proposed algorithm is illustrated in decomposition of difficult tensors which are associated with multiplication of two matrices.