A CDF-First Framework for Free-Form Density Estimation

TL;DR

This paper introduces a novel CDF-first framework for free-form density estimation that improves stability and expressivity by modeling the cumulative distribution function instead of the probability density function directly.

Contribution

It proposes a new approach that estimates the CDF with a Smooth Min-Max network, ensuring valid PDFs and better handling complex distributions, outperforming existing methods.

Findings

Outperforms state-of-the-art density estimators on various tasks.

Guarantees valid PDFs by construction.

Handles complex, multimodal distributions effectively.

Abstract

Conditional density estimation (CDE) is a fundamental task in machine learning that aims to model the full conditional law $\mathbb{P}(\mathbf{y} \mid \mathbf{x})$, beyond mere point prediction (e.g., mean, mode). A core challenge is free-form density estimation, capturing distributions that exhibit multimodality, asymmetry, or topological complexity without restrictive assumptions. However, prevailing methods typically estimate the probability density function (PDF) directly, which is mathematically ill-posed: differentiating the empirical distribution amplifies random fluctuations inherent in finite datasets, necessitating strong inductive biases that limit expressivity and fail when violated. We propose a CDF-first framework that circumvents this issue by estimating the cumulative distribution function (CDF), a stable and well-posed target, and then recovering the PDF via differentiation of the learned smooth CDF. Parameterizing the CDF with a Smooth Min-Max (SMM) network, our framework guarantees valid PDFs by construction, enables tractable approximate likelihood training, and preserves complex distributional shapes. For multivariate outputs, we use an autoregressive decomposition with SMM factors. Experiments demonstrate our approach outperforms state-of-the-art density estimators on a range of univariate and multivariate tasks.