Adaptive joint distribution learning

TL;DR

This paper introduces a novel framework for estimating joint probability distributions using tensor product RKHS, capable of handling large datasets efficiently and producing well-defined conditional distributions.

Contribution

It presents a new RKHS-based method for joint distribution estimation that is scalable, normalized, positive, and applicable to various learning tasks.

Findings

Efficient estimation of joint distributions from millions of samples.

Generation of well-defined normalized and positive conditional distributions.

Favorable numerical results demonstrating the method's effectiveness.

Abstract

We develop a new framework for estimating joint probability distributions using tensor product reproducing kernel Hilbert spaces (RKHS). Our framework accommodates a low-dimensional, normalized and positive model of a Radon--Nikodym derivative, which we estimate from sample sizes of up to several millions, alleviating the inherent limitations of RKHS modeling. Well-defined normalized and positive conditional distributions are natural by-products to our approach. Our proposal is fast to compute and accommodates learning problems ranging from prediction to classification. Our theoretical findings are supplemented by favorable numerical results.