Getting more from your regression model: A free lunch?

TL;DR

This paper introduces a simple method to extract detailed conditional distribution information from standard regression models, showing it performs well for quantile estimation and rivals specialized methods across various datasets.

Contribution

The authors propose a straightforward approach to approximate conditional distributions from standard regression outputs, enhancing quantile regression without additional complex modeling.

Findings

Method performs well for tail probability estimation

Comparable to quantile regression forest models

Effective across diverse benchmark datasets

Abstract

We consider a simple approach for approximating detailed information about the conditional distribution of a real-valued response variable, given values for its covariates, using only the outputs from a standard regression model. We validate this approach by assessing its performance in the context of quantile regression; when applied to the outputs of linear, gradient boosted tree ensemble and random forest models. We find that it compares favourably to the standard approach for estimating quantile regression functions, especially for commonly selected tail probabilities, and is highly competitive with the quantile regression forest model, across a large collection of benchmark data sets.