Graphical model for factorization and completion of relatively high rank tensors by sparse sampling

TL;DR

This paper introduces a tensor factorization method using sparse measurements on high-rank tensors with a focus on graph-based sampling, providing theoretical analysis and message-passing algorithms for data completion tasks.

Contribution

It proposes a novel tensor factorization framework with sparse, graph-structured measurements, along with theoretical insights and algorithms for high-dimensional inference.

Findings

Message-passing algorithms perform well in Bayes optimal settings.

Replica theory provides insights into inference performance in dense graph limits.

The approach is applicable to data completion in recommendation systems.

Abstract

We consider tensor factorizations based on sparse measurements of the components of relatively high rank tensors. The measurements are designed in a way that the underlying graph of interactions is a random graph. The setup will be useful in cases where a substantial amount of data is missing, as in completion of relatively high rank matrices for recommendation systems heavily used in social network services. In order to obtain theoretical insights on the setup, we consider statistical inference of the tensor factorization in a high dimensional limit, which we call as dense limit, where the graphs are large and dense but not fully connected. We build message-passing algorithms and test them in a Bayes optimal teacher-student setting in some specific cases. We also develop a replica theory to examine the performance of statistical inference in the dense limit based on a cumulant expansion. The latter approach allows one to avoid blind usage of Gaussian ansatz which fails in some fully connected systems.