Efficient, Anytime Algorithms for Calibration with Isotonic Regression under Strictly Convex Losses

TL;DR

This paper introduces efficient, online algorithms for calibrating estimators using isotonic regression under strictly convex losses, ensuring optimal monotone transforms with linear space and near-linear time complexity.

Contribution

It provides a novel, linear-time algorithm for finding optimal monotone calibrations under strictly convex losses, including an anytime version for streaming data.

Findings

Optimal monotone transforms are staircase functions.

The algorithms operate in linear space and near-linear time.

The methods are applicable in online, streaming settings.

Abstract

We investigate the calibration of estimations to increase performance with an optimal monotone transform on the estimator outputs. We start by studying the traditional square error setting with its weighted variant and show that the optimal monotone transform is in the form of a unique staircase function. We further show that this staircase behavior is preserved for general strictly convex loss functions. Their optimal monotone transforms are also unique, i.e., there exist a single staircase transform that achieves the minimum loss. We propose a linear time and space algorithm that can find such optimal transforms for specific loss settings. Our algorithm has an online implementation where the optimal transform for the samples observed so far are found in linear space and amortized time when the samples arrive in an ordered fashion. We also extend our results to cases where the functions are not trivial to individually optimize and propose an anytime algorithm, which has linear space and pseudo-linearithmic time complexity.