Brenier Isotonic Regression

TL;DR

This paper introduces Brenier isotonic regression, a novel multi-output regression method enforcing cyclic monotonicity using optimal transport theory, with applications in probability calibration and generalized linear models.

Contribution

It extends isotonic regression to multi-output settings by leveraging optimal transport to enforce cyclic monotonicity, connecting regression functions with convex potentials.

Findings

IR outperforms baseline methods in probability calibration.

The approach effectively enforces cyclic monotonicity in multi-output regression.

Demonstrates applicability in generalized linear models.

Abstract

Isotonic regression (IR) is shape-constrained regression to maintain a univariate fitting curve non-decreasing, which has numerous applications including single-index models and probability calibration. When it comes to multi-output regression, the classical IR is no longer applicable because the monotonicity is not readily extendable. We consider a novel multi-output regression problem where a regression function is \emph{cyclically monotone}. Roughly speaking, a cyclically monotone function is the gradient of some convex potential. Whereas enforcing cyclic monotonicity is apparently challenging, we leverage the fact that Kantorovich's optimal transport (OT) always yields a cyclically monotone coupling as an optimal solution. This perspective naturally allows us to interpret a regression function and the convex potential as a link function in generalized linear models and Brenier's potential in OT, respectively, and hence we call this IR extension \emph{Brenier isotonic regression}. We demonstrate experiments with probability calibration and generalized linear models. In particular, IR outperforms many famous baselines in probability calibration robustly.