Random forest estimation of conditional distribution functions and conditional quantiles

TL;DR

This paper provides a theoretical analysis of two random forest-based estimators for conditional distribution functions and quantiles, proving their almost sure consistency and demonstrating their application on a numerical example.

Contribution

It introduces the first proof of consistency for these estimators that incorporates bootstrap sampling in the analysis.

Findings

Both estimators are proven to be consistent uniformly almost surely.

The study includes a numerical example demonstrating the estimators in practice.

It is the first work to establish consistency proofs including bootstrap methods.

Abstract

We propose a theoretical study of two realistic estimators of conditional distribution functions and conditional quantiles using random forests. The estimation process uses the bootstrap samples generated from the original dataset when constructing the forest. Bootstrap samples are reused to define the first estimator, while the second requires only the original sample, once the forest has been built. We prove that both proposed estimators of the conditional distribution functions are consistent uniformly a.s. To the best of our knowledge, it is the first proof of consistency including the bootstrap part. We also illustrate the estimation procedures on a numerical example.