Recovery of Joint Probability Distribution from one-way marginals: Low rank Tensors and Random Projections

TL;DR

This paper introduces a novel method combining low-rank tensor decomposition and random projections to estimate joint probability distributions from one-way marginals, reducing the need for high-dimensional data.

Contribution

It presents a new algorithm that recovers joint probability tensors from one-way marginals using low-rank tensor techniques and random projections, linking tomography ideas with probabilistic modeling.

Findings

Successful estimation of joint distributions from one-way marginals.

Effective classification using MAP inference on estimated models.

Validated approach on synthetic and real-world datasets.

Abstract

Joint probability mass function (PMF) estimation is a fundamental machine learning problem. The number of free parameters scales exponentially with respect to the number of random variables. Hence, most work on nonparametric PMF estimation is based on some structural assumptions such as clique factorization adopted by probabilistic graphical models, imposition of low rank on the joint probability tensor and reconstruction from 3-way or 2-way marginals, etc. In the present work, we link random projections of data to the problem of PMF estimation using ideas from tomography. We integrate this idea with the idea of low-rank tensor decomposition to show that we can estimate the joint density from just one-way marginals in a transformed space. We provide a novel algorithm for recovering factors of the tensor from one-way marginals, test it across a variety of synthetic and real-world datasets, and also perform MAP inference on the estimated model for classification.