Efficient Algorithms for Non-convex Isotonic Regression through Submodular Optimization

TL;DR

This paper introduces efficient algorithms for non-convex isotonic regression by leveraging submodular optimization and stochastic dominance, demonstrating robustness to outliers and improved computational efficiency.

Contribution

It reformulates isotonic regression as a convex optimization problem on measures, proposing new discretization schemes and algorithms that handle non-convex loss functions effectively.

Findings

Algorithms are computationally efficient and scalable.

Non-convex loss functions improve robustness to outliers.

Proposed methods outperform existing approaches in experiments.

Abstract

We consider the minimization of submodular functions subject to ordering constraints. We show that this optimization problem can be cast as a convex optimization problem on a space of uni-dimensional measures, with ordering constraints corresponding to first-order stochastic dominance. We propose new discretization schemes that lead to simple and efficient algorithms based on zero-th, first, or higher order oracles; these algorithms also lead to improvements without isotonic constraints. Finally, our experiments show that non-convex loss functions can be much more robust to outliers for isotonic regression, while still leading to an efficient optimization problem.