Private Isotonic Regression

TL;DR

This paper develops differentially private algorithms for isotonic regression over general posets, providing near-optimal bounds on excess risk and efficient algorithms for special cases, advancing privacy-preserving statistical methods.

Contribution

It introduces the first pure-DP algorithms for general isotonic regression with tight bounds, and efficient algorithms for totally ordered sets with structural extensions.

Findings

Expected excess risk scales with poset width and size, inversely with sample size.

Lower bounds match upper bounds, showing optimality of the algorithms.

Efficient near-linear time algorithms for totally ordered sets with common loss functions.

Abstract

In this paper, we consider the problem of differentially private (DP) algorithms for isotonic regression. For the most general problem of isotonic regression over a partially ordered set (poset) $\mathcal{X}$ and for any Lipschitz loss function, we obtain a pure-DP algorithm that, given $n$ input points, has an expected excess empirical risk of roughly $\mathrm{width}(\mathcal{X}) \cdot \log|\mathcal{X}| / n$, where $\mathrm{width}(\mathcal{X})$ is the width of the poset. In contrast, we also obtain a near-matching lower bound of roughly $(\mathrm{width}(\mathcal{X}) + \log |\mathcal{X}|) / n$, that holds even for approximate-DP algorithms. Moreover, we show that the above bounds are essentially the best that can be obtained without utilizing any further structure of the poset. In the special case of a totally ordered set and for $\ell_1$ and $\ell_2^2$ losses, our algorithm can be implemented in near-linear running time; we also provide extensions of this algorithm to the problem of private isotonic regression with additional structural constraints on the output function.