Bin-wise Temperature Scaling (BTS): Improvement in Confidence Calibration Performance through Simple Scaling Techniques

TL;DR

This paper introduces Bin-wise Temperature Scaling (BTS), a simple yet effective method to improve the confidence calibration of neural networks by applying localized scaling techniques, enhancing reliability in critical applications.

Contribution

It proposes a novel bin-wise temperature scaling method combined with validation sample augmentation, significantly improving calibration across datasets and models.

Findings

Consistent calibration improvements across multiple datasets.

Enhanced confidence reliability in safety-critical applications.

Simple post-processing method with broad applicability.

Abstract

The prediction reliability of neural networks is important in many applications. Specifically, in safety-critical domains, such as cancer prediction or autonomous driving, a reliable confidence of model's prediction is critical for the interpretation of the results. Modern deep neural networks have achieved a significant improvement in performance for many different image classification tasks. However, these networks tend to be poorly calibrated in terms of output confidence. Temperature scaling is an efficient post-processing-based calibration scheme and obtains well calibrated results. In this study, we leverage the concept of temperature scaling to build a sophisticated bin-wise scaling. Furthermore, we adopt augmentation of validation samples for elaborated scaling. The proposed methods consistently improve calibration performance with various datasets and deep convolutional neural network models.