Deep Neural Network Calibration by Reducing Classifier Shift with Stochastic Masking

TL;DR

This paper introduces MaC-Cal, a stochastic masking method that improves deep neural network calibration by reducing classifier shift, especially under data corruption, leading to more reliable confidence estimates in critical applications.

Contribution

MaC-Cal is a novel mask-based calibration approach using stochastic sparsity and adaptive training to enhance confidence-accuracy alignment in DNNs.

Findings

MaC-Cal outperforms existing calibration methods in accuracy and robustness.

It effectively reduces calibration errors caused by underconfidence.

Demonstrates improved reliability under data corruption scenarios.

Abstract

In recent years, deep neural networks (DNNs) have shown competitive results in many fields. Despite this success, they often suffer from poor calibration, especially in safety-critical scenarios such as autonomous driving and healthcare, where unreliable confidence estimates can lead to serious consequences. Recent studies have focused on improving calibration by modifying the classifier, yet such efforts remain limited. Moreover, most existing approaches overlook calibration errors caused by underconfidence, which can be equally detrimental. To address these challenges, we propose MaC-Cal, a novel mask-based classifier calibration method that leverages stochastic sparsity to enhance the alignment between confidence and accuracy. MaC-Cal adopts a two-stage training scheme with adaptive sparsity, dynamically adjusting mask retention rates based on the deviation between confidence and accuracy. Extensive experiments show that MaC-Cal achieves superior calibration performance and robustness under data corruption, offering a practical and effective solution for reliable confidence estimation in DNNs.