Soft Contamination Means Benchmarks Test Shallow Generalization

TL;DR

This paper investigates how semantic duplicates in training data, known as soft contamination, bias benchmark performance estimates and confound genuine out-of-distribution generalization improvements.

Contribution

It highlights the widespread presence of semantic duplicates in training data, demonstrating their impact on benchmark results and questioning the validity of recent performance gains.

Findings

Semantic duplicates found in 78% of CodeForces data

Including duplicates improves benchmark scores

Performance on held-out data also improves with duplicates

Abstract

If LLM training data is polluted with benchmark test data, then benchmark performance gives biased estimates of out-of-distribution (OOD) generalization. Typical decontamination filters use n-gram matching which fail to detect semantic duplicates: sentences with equivalent (or near-equivalent) content that are not close in string space. We study this soft contamination of training data by semantic duplicates. Among other experiments, we embed the Olmo3 training corpus and find that: 1) contamination remains widespread, e.g. we find semantic duplicates for 78% of CodeForces and exact duplicates for 50% of ZebraLogic problems; 2) including semantic duplicates of benchmark data in training does improve benchmark performance; and 3) when finetuning on duplicates of benchmark datapoints, performance also improves on truly-held-out datapoints from the same benchmark. We argue that recent benchmark gains are thus confounded: the prevalence of soft contamination means gains reflect both genuine capability improvements and the accumulation of test data and effective test data in growing training corpora.