CLUE: Neural Networks Calibration via Learning Uncertainty-Error alignment

TL;DR

CLUE introduces a training method that aligns neural network uncertainty estimates with actual errors, improving calibration across diverse tasks and domains without extra computational costs.

Contribution

The paper presents CLUE, a novel differentiable loss function that explicitly aligns uncertainty with error during training, enhancing calibration and generalization.

Findings

CLUE achieves superior calibration across vision, regression, and language tasks.

It maintains competitive predictive performance compared to state-of-the-art methods.

The approach is domain-agnostic and computationally efficient.

Abstract

Reliable uncertainty estimation is critical for deploying neural networks (NNs) in real-world applications. While existing calibration techniques often rely on post-hoc adjustments or coarse-grained binning methods, they remain limited in scalability, differentiability, and generalization across domains. In this work, we introduce CLUE (Calibration via Learning Uncertainty-Error Alignment), a novel approach that explicitly aligns predicted uncertainty with observed error during training, grounded in the principle that well-calibrated models should produce uncertainty estimates that match their empirical loss. CLUE adopts a novel loss function that jointly optimizes predictive performance and calibration, using summary statistics of uncertainty and loss as proxies. The proposed method is fully differentiable, domain-agnostic, and compatible with standard training pipelines. Through extensive experiments on vision, regression, and language modeling tasks, including out-of-distribution and domain-shift scenarios, we demonstrate that CLUE achieves superior calibration quality and competitive predictive performance with respect to state-of-the-art approaches without imposing significant computational overhead.