Optimal solutions to the isotonic regression problem

TL;DR

This paper characterizes the optimal solutions for isotonic regression across various functionals, extending classical results to partial orders and demonstrating the optimality of pool-adjacent-violators solutions.

Contribution

It generalizes isotonic regression solutions for a broad class of functionals and extends results from total to partial order cases.

Findings

Solutions are optimal for all loss functions with the specified functional as the Bayes act.

Pool-adjacent-violators algorithm yields optimal solutions for total orders.

Simultaneous optimality does not hold in unimodal regression.

Abstract

In general, the solution to a regression problem is the minimizer of a given loss criterion, and depends on the specified loss function. The nonparametric isotonic regression problem is special, in that optimal solutions can be found by solely specifying a functional. These solutions will then be minimizers under all loss functions simultaneously as long as the loss functions have the requested functional as the Bayes act. For the functional, the only requirement is that it can be defined via an identification function, with examples including the expectation, quantile, and expectile functionals. Generalizing classical results, we characterize the optimal solutions to the isotonic regression problem for such functionals, and extend the results from the case of totally ordered explanatory variables to partial orders. For total orders, we show that any solution resulting from the pool-adjacent-violators algorithm is optimal. It is noteworthy, that simultaneous optimality is unattainable in the unimodal regression problem, despite its close connection.