Language model developers should report train-test overlap

TL;DR

This paper emphasizes the importance of transparency in reporting train-test overlap in language models, highlighting current deficiencies and advocating for standardized reporting to improve evaluation trust.

Contribution

It documents current practices of model developers regarding train-test overlap and advocates for mandatory reporting of overlap statistics for transparency.

Findings

Only 9 out of 30 developers report train-test overlap.

Open-source training data enables direct measurement of overlap.

Publishing overlap statistics increases transparency and trust.

Abstract

Language models are extensively evaluated, but correctly interpreting evaluation results requires knowledge of train-test overlap which refers to the extent to which the language model is trained on the very data it is being tested on. The public currently lacks adequate information about train-test overlap: most models have no public train-test overlap statistics, and third parties cannot directly measure train-test overlap since they do not have access to the training data. To make this clear, we document the practices of 30 model developers, finding that just 9 developers report train-test overlap: 4 developers release training data under open-source licenses, enabling the community to directly measure train-test overlap, and 5 developers publish their train-test overlap methodology and statistics. By engaging with language model developers, we provide novel information about train-test overlap for three additional developers. Overall, we take the position that language model developers should publish train-test overlap statistics and/or training data whenever they report evaluation results on public test sets. We hope our work increases transparency into train-test overlap to increase the community-wide trust in model evaluations.