Completing a joint PMF from projections: a low-rank coupled tensor factorization approach

TL;DR

This paper introduces a low-rank coupled tensor factorization method to reconstruct high-dimensional joint probability mass functions from low-dimensional projections, with guarantees of identifiability and practical algorithms demonstrated on rating prediction.

Contribution

It presents a novel approach for recovering joint PMFs from marginalized data using coupled low-rank tensor factorization, with theoretical guarantees and an algorithmic solution.

Findings

Guaranteed identifiability for low-rank joint PMFs

Effective approximation for higher-rank cases

Successful application to rating prediction

Abstract

There has recently been considerable interest in completing a low-rank matrix or tensor given only a small fraction (or few linear combinations) of its entries. Related approaches have found considerable success in the area of recommender systems, under machine learning. From a statistical estimation point of view, the gold standard is to have access to the joint probability distribution of all pertinent random variables, from which any desired optimal estimator can be readily derived. In practice high-dimensional joint distributions are very hard to estimate, and only estimates of low-dimensional projections may be available. We show that it is possible to identify higher-order joint PMFs from lower-order marginalized PMFs using coupled low-rank tensor factorization. Our approach features guaranteed identifiability when the full joint PMF is of low-enough rank, and effective approximation otherwise. We provide an algorithmic approach to compute the sought factors, and illustrate the merits of our approach using rating prediction as an example.