Nonparametric Probabilistic Regression with Coarse Learners

TL;DR

This paper introduces a nonparametric probabilistic regression method that combines coarse classifiers trained on different target coarsenings to produce accurate conditional density estimates with minimal assumptions, validated on various datasets.

Contribution

The paper proposes a novel nonparametric approach that integrates multiple coarse classifiers to estimate full probability densities for regression tasks, enhancing flexibility and interpretability.

Findings

Achieves high-fidelity prediction intervals in practice

Demonstrates competitive performance on large datasets

Provides insights through density examination on individual observations

Abstract

Probabilistic Regression refers to predicting a full probability density function for the target conditional on the features. We present a nonparametric approach to this problem which combines base classifiers (typically gradient boosted forests) trained on different coarsenings of the target value. By combining such classifiers and averaging the resulting densities, we are able to compute precise conditional densities with minimal assumptions on the shape or form of the density. We combine this approach with a structured cross-entropy loss function which serves to regularize and smooth the resulting densities. Prediction intervals computed from these densities are shown to have high fidelity in practice. Furthermore, examining the properties of these densities on particular observations can provide valuable insight. We demonstrate this approach on a variety of datasets and show competitive performance, particularly on larger datasets.