Neural Generative Distributional Regression

TL;DR

This paper introduces a neural network-based method to estimate conditional distributions by transforming known noise, enabling flexible modeling and applications like predictive intervals and density estimation.

Contribution

It proposes a novel estimator minimizing empirical energy distance to learn the generative transformation, with theoretical guarantees and practical effectiveness.

Findings

Estimator achieves adaptive nonparametric rates.

Method effectively models complex conditional distributions.

Numerical and real data demonstrate practical success.

Abstract

Any continuous conditional distribution of $Y$ given $X$ can be generated from a transform of a known noise distribution $U$ such as the uniform or normal distribution via $Y = g(X, U)$. This paper provides an estimator of such a generative transformation $g$ by minimizing the empirical energy distance between distributions of $Y$ and $g(X, U)$, and implements it via neural networks. The estimated distribution can then be readily applied to downstream tasks such as conditional moment estimation, predictive interval construction, and conditional density estimation. By leveraging the representation power of neural networks, the estimator can adaptively exploit low-dimensional structures in a purely algorithmic manner. Theoretically, we establish an oracle inequality attaining the adaptive optimal nonparametric rates. Numerical simulations and real data analysis further demonstrate the practical effectiveness of the proposed method.