Towards Robust Classification with Deep Generative Forests

TL;DR

This paper introduces Generative Forests, a deep probabilistic extension of Random Forests, enabling uncertainty estimation and out-of-distribution detection for more robust classification in tabular data.

Contribution

It presents Generative Forests (GeFs), a novel model that extends Random Forests to a generative framework for improved uncertainty quantification.

Findings

GeFs can measure prediction robustness.

GeFs effectively detect out-of-distribution samples.

GeFs outperform traditional models in uncertainty estimation.

Abstract

Decision Trees and Random Forests are among the most widely used machine learning models, and often achieve state-of-the-art performance in tabular, domain-agnostic datasets. Nonetheless, being primarily discriminative models they lack principled methods to manipulate the uncertainty of predictions. In this paper, we exploit Generative Forests (GeFs), a recent class of deep probabilistic models that addresses these issues by extending Random Forests to generative models representing the full joint distribution over the feature space. We demonstrate that GeFs are uncertainty-aware classifiers, capable of measuring the robustness of each prediction as well as detecting out-of-distribution samples.