Accurate Prediction and Uncertainty Estimation using Decoupled Prediction Interval Networks

TL;DR

This paper introduces a decoupled network architecture for regression that simultaneously achieves high prediction accuracy and reliable uncertainty estimation, outperforming existing methods on synthetic and real datasets.

Contribution

A novel two-stage training process for prediction and uncertainty estimation that improves accuracy and coverage simultaneously.

Findings

Reduces prediction error by 23-34% on benchmarks.

Maintains 95% coverage probability in most datasets.

Enhances active learning performance with 17-36% error reduction.

Abstract

We propose a network architecture capable of reliably estimating uncertainty of regression based predictions without sacrificing accuracy. The current state-of-the-art uncertainty algorithms either fall short of achieving prediction accuracy comparable to the mean square error optimization or underestimate the variance of network predictions. We propose a decoupled network architecture that is capable of accomplishing both at the same time. We achieve this by breaking down the learning of prediction and prediction interval (PI) estimations into a two-stage training process. We use a custom loss function for learning a PI range around optimized mean estimation with a desired coverage of a proportion of the target labels within the PI range. We compare the proposed method with current state-of-the-art uncertainty quantification algorithms on synthetic datasets and UCI benchmarks, reducing the error in the predictions by 23 to 34% while maintaining 95% Prediction Interval Coverage Probability (PICP) for 7 out of 9 UCI benchmark datasets. We also examine the quality of our predictive uncertainty by evaluating on Active Learning and demonstrating 17 to 36% error reduction on UCI benchmarks.