Learning with Monotone Adversarial Corruptions

TL;DR

This paper introduces a monotone adversarial corruption model revealing that standard optimal learning algorithms for binary classification can fail under such corruptions, highlighting their dependence on data exchangeability.

Contribution

It demonstrates that optimal algorithms can become suboptimal with monotone corruptions, while uniform convergence algorithms remain robust, exposing limitations of current methods.

Findings

Optimal algorithms degrade under monotone corruptions

Uniform convergence algorithms maintain guarantees

Overreliance on data exchangeability is exposed

Abstract

We study the extent to which standard machine learning algorithms rely on exchangeability and independence of data by introducing a monotone adversarial corruption model. In this model, an adversary, upon looking at a "clean" i.i.d. dataset, inserts additional "corrupted" points of their choice into the dataset. These added points are constrained to be monotone corruptions, in that they get labeled according to the ground-truth target function. Perhaps surprisingly, we demonstrate that in this setting, all known optimal learning algorithms for binary classification can be made to achieve suboptimal expected error on a new independent test point drawn from the same distribution as the clean dataset. On the other hand, we show that uniform convergence-based algorithms do not degrade in their guarantees. Our results showcase how optimal learning algorithms break down in the face of seemingly helpful monotone corruptions, exposing their overreliance on exchangeability.