Consensus-based In-Network Computation of the PARAFAC Decomposition

TL;DR

This paper introduces distributed algorithms for computing the PARAFAC tensor decomposition across a network, enabling collaboration among nodes to estimate global and local factors even with limited local data.

Contribution

It proposes novel distributed ALS and LM algorithms that leverage consensus methods for in-network tensor decomposition, addressing local data limitations.

Findings

Algorithms emulate centralized performance with ideal data exchange.

Distributed methods successfully estimate global and local factors.

Performance evaluated under ideal and imperfect conditions.

Abstract

In this work, we present a new approach for the distributed computation of the PARAFAC decomposition of a third-order tensor across a network of collaborating nodes. We are interested in the case where the overall data gathered across the network can be modeled as a data tensor admitting an essentially unique PARAFAC decomposition, while each node only observes a sub-tensor with not necessarily enough diversity so that identifiability conditions are not locally fulfilled at each node. In this situation, conventional (centralized) tensor based methods cannot be applied individually at each node. By allowing collaboration between neighboring nodes of the network, we propose distributed versions of the alternating least squares (ALS) and Levenberg-Marquardt (LM) algorithms for the in-network estimation of the factor matrices of a third-order tensor. We assume that one of the factor matrices contains parameters that are local to each node, while the two remaining factor matrices contain global parameters that are common to the whole network. The proposed algorithms combine the estimation of the local factors with an in-network computation of the global factors of the PARAFAC decomposition using average consensus over graphs. They emulate their centralized counterparts in the case of ideal data exchange and ideal consensus computations. The performance of the proposed algorithms are evaluated in both ideal and imperfect cases.