Beyond Tsybakov: Model Margin Noise and $\mathcal{H}$-Consistency Bounds

TL;DR

This paper introduces the Model Margin Noise condition, a weaker alternative to Tsybakov noise, leading to improved $ ext{H}$-consistency bounds for classification tasks, especially in intermediate noise regimes.

Contribution

It proposes the Model Margin Noise assumption, extending $ ext{H}$-consistency bounds under weaker conditions than Tsybakov noise, applicable to both binary and multi-class classification.

Findings

Model Margin Noise is weaker than Tsybakov noise.

Enhanced $ ext{H}$-consistency bounds are derived under MM noise.

Bounds interpolate between linear and square-root regimes.

Abstract

We introduce a new low-noise condition for classification, the Model Margin Noise (MM noise) assumption, and derive enhanced $\mathcal{H}$-consistency bounds under this condition. MM noise is weaker than Tsybakov noise condition: it is implied by Tsybakov noise condition but can hold even when Tsybakov fails, because it depends on the discrepancy between a given hypothesis and the Bayes-classifier rather than on the intrinsic distributional minimal margin (see Figure 1 for an illustration of an explicit example). This hypothesis-dependent assumption yields enhanced $\mathcal{H}$-consistency bounds for both binary and multi-class classification. Our results extend the enhanced $\mathcal{H}$-consistency bounds of Mao, Mohri, and Zhong (2025a) with the same favorable exponents but under a weaker assumption than the Tsybakov noise condition; they interpolate smoothly between linear and square-root regimes for intermediate noise levels. We also instantiate these bounds for common surrogate loss families and provide illustrative tables.