Uncertain Trees: Dealing with Uncertain Inputs in Regression Trees

TL;DR

This paper introduces a novel extension of regression trees that accounts for uncertainties in input variables, allowing for probabilistic region membership and improving robustness in practical applications.

Contribution

It presents the first method to incorporate input uncertainties into regression trees, adapting standard quadratic loss optimization to probabilistic input regions.

Findings

Effective handling of input uncertainties demonstrated on multiple datasets.

Improved regression accuracy with uncertain inputs compared to traditional trees.

Probabilistic region assignment enhances model robustness.

Abstract

Tree-based ensemble methods, as Random Forests and Gradient Boosted Trees, have been successfully used for regression in many applications and research studies. Furthermore, these methods have been extended in order to deal with uncertainty in the output variable, using for example a quantile loss in Random Forests (Meinshausen, 2006). To the best of our knowledge, no extension has been provided yet for dealing with uncertainties in the input variables, even though such uncertainties are common in practical situations. We propose here such an extension by showing how standard regression trees optimizing a quadratic loss can be adapted and learned while taking into account the uncertainties in the inputs. By doing so, one no longer assumes that an observation lies into a single region of the regression tree, but rather that it belongs to each region with a certain probability. Experiments conducted on several data sets illustrate the good behavior of the proposed extension.