Distributional regression: CRPS-error bounds for model fitting, model selection and convex aggregation

TL;DR

This paper establishes theoretical bounds for distributional regression models using CRPS, aiding in model fitting, selection, and aggregation, with practical applications demonstrated on real datasets.

Contribution

It provides novel concentration bounds for CRPS-based estimation error, model selection regret, and convex aggregation, applicable to various distributional regression models.

Findings

Derived concentration bounds for estimation error and regret.

Applied theoretical results to models like EMOS, neural networks, and random forests.

Validated findings on QSAR toxicity and airfoil noise datasets.

Abstract

Distributional regression aims at estimating the conditional distribution of a targetvariable given explanatory co-variates. It is a crucial tool for forecasting whena precise uncertainty quantification is required. A popular methodology consistsin fitting a parametric model via empirical risk minimization where the risk ismeasured by the Continuous Rank Probability Score (CRPS). For independentand identically distributed observations, we provide a concentration result for theestimation error and an upper bound for its expectation. Furthermore, we considermodel selection performed by minimization of the validation error and provide aconcentration bound for the regret. A similar result is proved for convex aggregationof models. Finally, we show that our results may be applied to various models suchas Ensemble Model Output Statistics (EMOS), distributional regression networks,distributional nearest neighbors or distributional random forests and we illustrateour findings on two data sets (QSAR aquatic toxicity and Airfoil self-noise).