High-Quality Prediction Intervals for Deep Learning: A Distribution-Free, Ensembled Approach

TL;DR

This paper introduces a distribution-free, ensembled neural network method for generating high-quality prediction intervals that are narrow yet reliably capture the true data distribution, improving uncertainty quantification in regression.

Contribution

It proposes a novel loss function derived from axioms for prediction intervals, enabling neural networks to produce better uncertainty estimates without distributional assumptions.

Findings

Reduces average prediction interval width by over 10%

Outperforms current state-of-the-art uncertainty methods

Effectively accounts for model uncertainty through ensembling

Abstract

This paper considers the generation of prediction intervals (PIs) by neural networks for quantifying uncertainty in regression tasks. It is axiomatic that high-quality PIs should be as narrow as possible, whilst capturing a specified portion of data. We derive a loss function directly from this axiom that requires no distributional assumption. We show how its form derives from a likelihood principle, that it can be used with gradient descent, and that model uncertainty is accounted for in ensembled form. Benchmark experiments show the method outperforms current state-of-the-art uncertainty quantification methods, reducing average PI width by over 10%.