The bias of isotonic regression

TL;DR

This paper provides a precise characterization of the bias in isotonic regression estimators, showing it scales with sample size depending on the smoothness of the underlying mean, under broad conditions.

Contribution

It offers the first sharp bias bounds for isotonic regression that depend on the smoothness of the true mean, requiring minimal assumptions.

Findings

Bias scales as $O(n^{-eta/3})$ up to log factors

Results hold under subexponential noise tails and strict monotonicity

No need for symmetric noise or restrictive assumptions

Abstract

We study the bias of the isotonic regression estimator. While there is extensive work characterizing the mean squared error of the isotonic regression estimator, relatively little is known about the bias. In this paper, we provide a sharp characterization, proving that the bias scales as $O(n^{-\beta/3})$ up to log factors, where $1 \leq \beta \leq 2$ is the exponent corresponding to H{\"o}lder smoothness of the underlying mean. Importantly, this result only requires a strictly monotone mean and that the noise distribution has subexponential tails, without relying on symmetric noise or other restrictive assumptions.