Probability Bounding: Post-Hoc Calibration via Box-Constrained Softmax

TL;DR

This paper introduces probability bounding, a post-hoc calibration method using box-constrained softmax to improve probability estimates of neural networks, with theoretical guarantees and efficient algorithms demonstrated on real datasets.

Contribution

The paper proposes a novel post-hoc calibration technique with a new box-constrained softmax function and provides theoretical guarantees and efficient algorithms for its computation.

Findings

Consistently reduces calibration errors on four real-world datasets.

Provides theoretical guarantees for the probability bounding method.

Introduces an efficient algorithm for computing the box-constrained softmax.

Abstract

Many studies have observed that modern neural networks achieve high accuracy while producing poorly calibrated probabilities, making calibration a critical practical issue. In this work, we propose probability bounding (PB), a novel post-hoc calibration method that mitigates both underconfidence and overconfidence by learning lower and upper bounds on the output probabilities. To implement PB, we introduce the box-constrained softmax (BCSoftmax) function, a generalization of Softmax that explicitly enforces lower and upper bounds on the output probabilities. While BCSoftmax is formulated as the solution to a box-constrained optimization problem, we develop an exact and efficient algorithm for computing BCSoftmax. We further provide theoretical guarantees for PB and introduce two variants of PB. We demonstrate the effectiveness of our methods experimentally on four real-world datasets, consistently reducing calibration errors. Our Python implementation is available at https://github.com/neonnnnn/torchbcsoftmax.