Multicalibration as Boosting for Regression

TL;DR

This paper establishes a connection between multicalibration and boosting in regression, introduces a simple boosting algorithm that guarantees Bayes optimality under weak assumptions, and demonstrates its empirical effectiveness.

Contribution

It provides a novel characterization of multicalibration via swap regret, and presents an agnostic boosting algorithm for regression that requires only a standard squared error oracle.

Findings

Algorithm converges to Bayes optimality without realizability assumptions.

Weak learning condition on class H is necessary and sufficient for multicalibration to imply Bayes optimality.

Empirical results demonstrate the effectiveness of the proposed boosting algorithm.

Abstract

We study the connection between multicalibration and boosting for squared error regression. First we prove a useful characterization of multicalibration in terms of a ``swap regret'' like condition on squared error. Using this characterization, we give an exceedingly simple algorithm that can be analyzed both as a boosting algorithm for regression and as a multicalibration algorithm for a class H that makes use only of a standard squared error regression oracle for H. We give a weak learning assumption on H that ensures convergence to Bayes optimality without the need to make any realizability assumptions -- giving us an agnostic boosting algorithm for regression. We then show that our weak learning assumption on H is both necessary and sufficient for multicalibration with respect to H to imply Bayes optimality. We also show that if H satisfies our weak learning condition relative to another class C then multicalibration with respect to H implies multicalibration with respect to C. Finally we investigate the empirical performance of our algorithm experimentally using an open source implementation that we make available. Our code repository can be found at https://github.com/Declancharrison/Level-Set-Boosting.