Class Adaptive Network Calibration

TL;DR

This paper introduces Class Adaptive Label Smoothing (CALS), a novel method that learns class-wise calibration multipliers during training, improving deep neural network calibration across various tasks and addressing limitations of fixed, class-agnostic penalties.

Contribution

The paper proposes CALS, an adaptive, class-specific calibration technique using an augmented Lagrangian approach, enhancing calibration without extensive hyper-parameter tuning.

Findings

CALS outperforms existing calibration methods on multiple benchmarks.

It effectively handles class imbalance and intrinsic class difficulty.

The method improves both calibration and accuracy across tasks.

Abstract

Recent studies have revealed that, beyond conventional accuracy, calibration should also be considered for training modern deep neural networks. To address miscalibration during learning, some methods have explored different penalty functions as part of the learning objective, alongside a standard classification loss, with a hyper-parameter controlling the relative contribution of each term. Nevertheless, these methods share two major drawbacks: 1) the scalar balancing weight is the same for all classes, hindering the ability to address different intrinsic difficulties or imbalance among classes; and 2) the balancing weight is usually fixed without an adaptive strategy, which may prevent from reaching the best compromise between accuracy and calibration, and requires hyper-parameter search for each application. We propose Class Adaptive Label Smoothing (CALS) for calibrating deep networks, which allows to learn class-wise multipliers during training, yielding a powerful alternative to common label smoothing penalties. Our method builds on a general Augmented Lagrangian approach, a well-established technique in constrained optimization, but we introduce several modifications to tailor it for large-scale, class-adaptive training. Comprehensive evaluation and multiple comparisons on a variety of benchmarks, including standard and long-tailed image classification, semantic segmentation, and text classification, demonstrate the superiority of the proposed method. The code is available at https://github.com/by-liu/CALS.