Ensembles of Probabilistic Regression Trees

TL;DR

This paper introduces ensemble probabilistic regression trees that offer smooth objective function approximations, proves their consistency, and compares their performance and bias-variance trade-offs with existing methods.

Contribution

It presents a novel ensemble approach for probabilistic regression trees, demonstrating their theoretical consistency and empirical advantages.

Findings

Ensemble probabilistic trees are consistent.

They achieve favorable bias-variance trade-offs.

They outperform state-of-the-art methods in prediction accuracy.

Abstract

Tree-based ensemble methods such as random forests, gradient-boosted trees, and Bayesianadditive regression trees have been successfully used for regression problems in many applicationsand research studies. In this paper, we study ensemble versions of probabilisticregression trees that provide smooth approximations of the objective function by assigningeach observation to each region with respect to a probability distribution. We prove thatthe ensemble versions of probabilistic regression trees considered are consistent, and experimentallystudy their bias-variance trade-off and compare them with the state-of-the-art interms of performance prediction.