Calibeating Made Simple

TL;DR

This paper introduces a unified approach to calibeating, a method for refining external forecasts online, by leveraging online learning techniques to achieve optimal rates across various loss functions and settings.

Contribution

It reduces calibeating to regret minimization, providing new optimal rates for multiple losses and extending to multi-calibeating and calibration simultaneously.

Findings

Achieves $O(\log T)$ calibeating rate for Brier and log losses.

Provides optimal rates for mixable and bounded losses.

First calibrated algorithm with optimal calibeating rate for binary predictions.

Abstract

We study calibeating, the problem of post-processing external forecasts online to minimize cumulative losses and match an informativeness-based benchmark. Unlike prior work, which analyzed calibeating for specific losses with specific arguments, we reduce calibeating to existing online learning techniques and obtain results for general proper losses. More concretely, we first show that calibeating is minimax-equivalent to regret minimization. This recovers the $O(\log T)$ calibeating rate of Foster and Hart [FH23] for the Brier and log losses and its optimality, and yields new optimal calibeating rates for mixable losses and general bounded losses. Second, we prove that multi-calibeating is minimax-equivalent to the combination of calibeating and the classical expert problem. This yields new optimal multi-calibeating rates for mixable losses, including Brier and log losses, and general bounded losses. Finally, we obtain new bounds for achieving calibeating and calibration simultaneously for the Brier loss. For binary predictions, our result gives the first calibrated algorithm that at the same time also achieves the optimal $O(\log T)$ calibeating rate.