Divide et Calibra: Multiclass Local Calibration via Vector Quantization

TL;DR

This paper introduces a novel multiclass calibration method using vector quantization to create region-specific calibration maps, improving local calibration accuracy while preserving global calibration and predictive performance.

Contribution

It proposes a structured, region-based calibration approach with shared codeword factors and Dirichlet parameterization, addressing limitations of existing global and local methods.

Findings

Significant improvements in local calibration on benchmark datasets

Maintains competitive global calibration and predictive accuracy

Learns heterogeneous calibration maps effective even in sparse regions

Abstract

Accurate and well-calibrated Machine Learning (ML) models are mandatory in high-stakes settings, yet effective multiclass calibration remains challenging: global approaches assume calibration errors are homogeneous across the latent space, while local methods often rely on latent-space dimensionality reduction, which leads to information loss. To address these issues, we propose a compositional approach to multiclass calibration, where region-specific calibration maps are constructed from shared codeword-dependent factors. We instantiate this idea via Vector Quantization (VQ), which induces a structured partition of the representation space, and an indexed parameterization of Dirichlet concentrations that enables parameter sharing across regions. Our approach learns heterogeneous calibration maps that generalize well even to sparse regions of the latent space. Experiments on benchmark datasets show significant improvements in local calibration while maintaining competitive global calibration and predictive performance.