Parameterized Temperature Scaling for Boosting the Expressive Power in Post-Hoc Uncertainty Calibration

TL;DR

This paper introduces Parametrized Temperature Scaling (PTS), a novel post-hoc calibration method that uses a neural network to compute prediction-specific temperatures, significantly improving calibration performance across various models and datasets.

Contribution

The paper proposes PTS, a new calibration technique that enhances the expressive power of temperature scaling by predicting individual temperatures with a neural network.

Findings

PTS outperforms existing calibration methods across multiple datasets.

Prediction-specific temperatures improve calibration accuracy.

The approach preserves model accuracy while enhancing calibration.

Abstract

We address the problem of uncertainty calibration and introduce a novel calibration method, Parametrized Temperature Scaling (PTS). Standard deep neural networks typically yield uncalibrated predictions, which can be transformed into calibrated confidence scores using post-hoc calibration methods. In this contribution, we demonstrate that the performance of accuracy-preserving state-of-the-art post-hoc calibrators is limited by their intrinsic expressive power. We generalize temperature scaling by computing prediction-specific temperatures, parameterized by a neural network. We show with extensive experiments that our novel accuracy-preserving approach consistently outperforms existing algorithms across a large number of model architectures, datasets and metrics.