Learning Half-Spaces from Perturbed Contrastive Examples

TL;DR

This paper analyzes a contrastive learning model where contrastive examples are perturbed by noise, demonstrating that such perturbations can improve learning efficiency for half-spaces and thresholds.

Contribution

It introduces a noise-perturbed contrastive example mechanism and characterizes its impact on sample complexity for learning half-spaces and thresholds.

Findings

Perturbed contrastive examples can accelerate learning.

The model's effectiveness depends on the noise function $f$.

Speed-up in learning is shown under certain conditions on $f$.

Abstract

We study learning under a two-step contrastive example oracle, as introduced by Mansouri et. al. (2025), where each queried (or sampled) labeled example is paired with an additional contrastive example of opposite label. While Mansouri et al. assume an idealized setting, where the contrastive example is at minimum distance of the originally queried/sampled point, we introduce and analyze a mechanism, parameterized by a non-decreasing noise function $f$, under which this ideal contrastive example is perturbed. The amount of perturbation is controlled by $f(d)$, where $d$ is the distance of the queried/sampled point to the decision boundary. Intuitively, this results in higher-quality contrastive examples for points closer to the decision boundary. We study this model in two settings: (i) when the maximum perturbation magnitude is fixed, and (ii) when it is stochastic. For one-dimensional thresholds and for half-spaces under the uniform distribution on a bounded domain, we characterize active and passive contrastive sample complexity in dependence on the function $f$. We show that, under certain conditions on $f$, the presence of contrastive examples speeds up learning in terms of asymptotic query complexity and asymptotic expected query complexity.