Accelerated Canonical Polyadic Decomposition by Using Mode Reduction

TL;DR

This paper introduces a new efficient CPD method that reduces high-order tensors to third-order tensors, avoiding frequent unfolding and improving scalability and speed for large-scale tensor analysis.

Contribution

The paper proposes a novel mode reduction technique converting Nth-order tensors to third-order, maintaining uniqueness and accuracy while enhancing computational efficiency.

Findings

The method is more efficient than existing CPD algorithms.

It better avoids local minima during tensor decomposition.

The approach preserves the essential uniqueness of the CPD.

Abstract

Canonical Polyadic (or CANDECOMP/PARAFAC, CP) decompositions (CPD) are widely applied to analyze high order tensors. Existing CPD methods use alternating least square (ALS) iterations and hence need to unfold tensors to each of the $N$ modes frequently, which is one major bottleneck of efficiency for large-scale data and especially when $N$ is large. To overcome this problem, in this paper we proposed a new CPD method which converts the original $N$th ($N>3$) order tensor to a 3rd-order tensor first. Then the full CPD is realized by decomposing this mode reduced tensor followed by a Khatri-Rao product projection procedure. This way is quite efficient as unfolding to each of the $N$ modes are avoided, and dimensionality reduction can also be easily incorporated to further improve the efficiency. We show that, under mild conditions, any $N$th-order CPD can be converted into a 3rd-order case but without destroying the essential uniqueness, and theoretically gives the same results as direct $N$-way CPD methods. Simulations show that, compared with state-of-the-art CPD methods, the proposed method is more efficient and escape from local solutions more easily.