Nonparametric estimation of conditional probability distributions using a generative approach based on conditional push-forward neural networks

TL;DR

This paper introduces CPFN, a neural network framework for estimating conditional distributions by learning a stochastic map, enabling efficient sampling and statistical estimation without complex training procedures.

Contribution

The paper proposes a novel conditional push-forward neural network framework that simplifies conditional distribution estimation and training, with theoretical consistency guarantees and competitive empirical performance.

Findings

CPFN achieves competitive or superior performance compared to state-of-the-art methods.

The model enables efficient conditional sampling and statistical estimation.

It is lightweight, easy to train, and does not require invertibility or adversarial training.

Abstract

We introduce conditional push-forward neural networks (CPFN), a generative framework for conditional distribution estimation. Instead of directly modeling the conditional density $f_{Y|X}$, CPFN learns a stochastic map $\varphi=\varphi(x,u)$ such that $\varphi(x,U)$ and $Y|X=x$ follow approximately the same law, with $U$ a suitable random vector of pre-defined latent variables. This enables efficient conditional sampling and straightforward estimation of conditional statistics through Monte Carlo methods. The model is trained via an objective function derived from a Kullback-Leibler formulation, without requiring invertibility or adversarial training. We establish a near-asymptotic consistency result and demonstrate experimentally that CPFN can achieve performance competitive with, or even superior to, state-of-the-art methods, including kernel estimators, tree-based algorithms, and popular deep learning techniques, all while remaining lightweight and easy to train.