Partition Tree: Conditional Density Estimation over General Outcome Spaces

TL;DR

Partition Tree introduces a scalable, nonparametric tree-based method for conditional density estimation over diverse outcome spaces, outperforming traditional probabilistic trees in prediction accuracy.

Contribution

It presents a unified framework modeling conditional densities with data-adaptive partitions, supporting continuous and categorical variables without parametric assumptions.

Findings

Outperforms CART-style trees in probabilistic prediction

Competitive with state-of-the-art probabilistic tree methods

Effective over general outcome spaces

Abstract

We propose Partition Tree, a novel tree-based framework for conditional density estimation over general outcome spaces that supports both continuous and categorical variables within a unified formulation. Our approach models conditional distributions as piecewise-constant densities on data-adaptive partitions and learns trees by directly minimizing conditional negative log-likelihood. This yields a scalable, nonparametric alternative to existing probabilistic trees that does not make parametric assumptions about the target distribution. We further introduce Partition Forest, a bagging extension obtained by averaging conditional densities. Empirically, we demonstrate improved probabilistic prediction over CART-style trees and competitive performance compared to state-of-the-art probabilistic tree methods and Random Forests.