Calibeating Made Simple
Yurong Chen, Zhiyi Huang, Michael I. Jordan, Haipeng Luo

TL;DR
This paper introduces a unified approach to calibeating, a method for refining external forecasts online, by leveraging online learning techniques to achieve optimal rates across various loss functions and settings.
Contribution
It reduces calibeating to regret minimization, providing new optimal rates for multiple losses and extending to multi-calibeating and calibration simultaneously.
Findings
Achieves $O(\log T)$ calibeating rate for Brier and log losses.
Provides optimal rates for mixable and bounded losses.
First calibrated algorithm with optimal calibeating rate for binary predictions.
Abstract
We study calibeating, the problem of post-processing external forecasts online to minimize cumulative losses and match an informativeness-based benchmark. Unlike prior work, which analyzed calibeating for specific losses with specific arguments, we reduce calibeating to existing online learning techniques and obtain results for general proper losses. More concretely, we first show that calibeating is minimax-equivalent to regret minimization. This recovers the calibeating rate of Foster and Hart [FH23] for the Brier and log losses and its optimality, and yields new optimal calibeating rates for mixable losses and general bounded losses. Second, we prove that multi-calibeating is minimax-equivalent to the combination of calibeating and the classical expert problem. This yields new optimal multi-calibeating rates for mixable losses, including Brier and log losses, and general…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Bandit Algorithms Research · Forecasting Techniques and Applications · Machine Learning and Algorithms
