Fast Nonparametric Conditional Density Estimation

TL;DR

This paper introduces a fast double kernel method for nonparametric conditional density estimation, enabling applications to large multivariate datasets with significant speed improvements.

Contribution

It develops a novel dual-tree algorithm for bandwidth selection in nonparametric conditional density estimation, significantly reducing computational costs.

Findings

Achieved up to 3.8 million times speedup in experiments.

Enabled application to large multivariate datasets, including SDSS redshift prediction.

First application of nonparametric conditional density estimation to high-dimensional data.

Abstract

Conditional density estimation generalizes regression by modeling a full density f(yjx) rather than only the expected value E(yjx). This is important for many tasks, including handling multi-modality and generating prediction intervals. Though fundamental and widely applicable, nonparametric conditional density estimators have received relatively little attention from statisticians and little or none from the machine learning community. None of that work has been applied to greater than bivariate data, presumably due to the computational difficulty of data-driven bandwidth selection. We describe the double kernel conditional density estimator and derive fast dual-tree-based algorithms for bandwidth selection using a maximum likelihood criterion. These techniques give speedups of up to 3.8 million in our experiments, and enable the first applications to previously intractable large multivariate datasets, including a redshift prediction problem from the Sloan Digital Sky Survey.