Learning Multivariate CDFs and Copulas using Tensor Factorization

TL;DR

This paper introduces a tensor factorization approach to learn multivariate CDFs and copulas, providing a scalable, identifiable, and efficient alternative to traditional density-based methods, with applications in regression, sampling, and imputation.

Contribution

It proposes a novel low-rank tensor decomposition method for multivariate CDFs and copulas, overcoming limitations of existing density-focused approaches and offering theoretical guarantees.

Findings

Superior performance on synthetic and real datasets

Efficient sampling and inference capabilities

Better density estimates via CDF models than PDF models

Abstract

Learning the multivariate distribution of data is a core challenge in statistics and machine learning. Traditional methods aim for the probability density function (PDF) and are limited by the curse of dimensionality. Modern neural methods are mostly based on black-box models, lacking identifiability guarantees. In this work, we aim to learn multivariate cumulative distribution functions (CDFs), as they can handle mixed random variables, allow efficient box probability evaluation, and have the potential to overcome local sample scarcity owing to their cumulative nature. We show that any grid sampled version of a joint CDF of mixed random variables admits a universal representation as a naive Bayes model via the Canonical Polyadic (tensor-rank) decomposition. By introducing a low-rank model, either directly in the raw data domain, or indirectly in a transformed (Copula) domain, the resulting model affords efficient sampling, closed form inference and uncertainty quantification, and comes with uniqueness guarantees under relatively mild conditions. We demonstrate the superior performance of the proposed model in several synthetic and real datasets and applications including regression, sampling and data imputation. Interestingly, our experiments with real data show that it is possible to obtain better density/mass estimates indirectly via a low-rank CDF model, than a low-rank PDF/PMF model.