Making Learners (More) Monotone

TL;DR

This paper introduces three algorithms that modify supervised learning models to ensure more monotonic performance, addressing non-monotonic behavior where more data doesn't always improve results.

Contribution

It proposes novel algorithms with theoretical guarantees to enforce monotonicity in learning models, validated on real datasets.

Findings

Less than 1% non-monotone decisions on MNIST

Algorithms maintain competitive error rates

High-probability guarantees for monotonicity

Abstract

Learning performance can show non-monotonic behavior. That is, more data does not necessarily lead to better models, even on average. We propose three algorithms that take a supervised learning model and make it perform more monotone. We prove consistency and monotonicity with high probability, and evaluate the algorithms on scenarios where non-monotone behaviour occurs. Our proposed algorithm $\text{MT}_{\text{HT}}$ makes less than $1\%$ non-monotone decisions on MNIST while staying competitive in terms of error rate compared to several baselines.