Improving Uncertainty Calibration via Prior Augmented Data

TL;DR

This paper introduces a method to improve the calibration of neural network predictions by adjusting overconfident regions towards a prior distribution, enhancing reliability across classification and regression tasks.

Contribution

The proposed approach identifies overconfident regions in feature space and adjusts their entropy towards the prior, applicable to any probabilistic neural network model.

Findings

Improved calibration in neural networks across tasks

Effective in both classification and regression

Model-agnostic and easy to integrate

Abstract

Neural networks have proven successful at learning from complex data distributions by acting as universal function approximators. However, they are often overconfident in their predictions, which leads to inaccurate and miscalibrated probabilistic predictions. The problem of overconfidence becomes especially apparent in cases where the test-time data distribution differs from that which was seen during training. We propose a solution to this problem by seeking out regions of feature space where the model is unjustifiably overconfident, and conditionally raising the entropy of those predictions towards that of the prior distribution of the labels. Our method results in a better calibrated network and is agnostic to the underlying model structure, so it can be applied to any neural network which produces a probability density as an output. We demonstrate the effectiveness of our method and validate its performance on both classification and regression problems, applying it to recent probabilistic neural network models.