Transforming Conditional Density Estimation Into a Single Nonparametric Regression Task

TL;DR

This paper introduces a novel approach to conditional density estimation by transforming it into a single regression task using auxiliary samples, enabling the use of powerful regression methods like neural networks.

Contribution

It presents a new method called condensité that leverages auxiliary samples for improved conditional density estimation and provides theoretical convergence guarantees.

Findings

Condensité achieves competitive or superior performance on synthetic, survey, and satellite datasets.

The method effectively captures complex conditional densities in high-dimensional settings.

Empirical results align with established findings, demonstrating practical applicability.

Abstract

We propose a way of transforming the problem of conditional density estimation into a single nonparametric regression task via the introduction of auxiliary samples. This allows leveraging regression methods that work well in high dimensions, such as neural networks and decision trees. Our main theoretical result characterizes and establishes the convergence of our estimator to the true conditional density in the data limit. We develop condensit\'e, a method that implements this approach. We demonstrate the benefit of the auxiliary samples on synthetic data and showcase that condensit\'e can achieve good out-of-the-box results. We evaluate our method on a large population survey dataset and on a satellite imaging dataset. In both cases, we find that condensit\'e matches or outperforms the state of the art and yields conditional densities in line with established findings in the literature on each dataset. Our contribution opens up new possibilities for regression-based conditional density estimation and the empirical results indicate strong promise for applied research.