Training-Induced Bias Toward LLM-Generated Content in Dense Retrieval

TL;DR

This paper investigates how training processes influence the bias of dense retrieval models towards LLM-generated content, revealing that such bias is training-induced rather than inherent, with implications for model training strategies.

Contribution

It provides a controlled evaluation showing that source bias in dense retrieval models depends on training data and procedures, especially fine-tuning on LLM-generated text.

Findings

Supervised fine-tuning on MS MARCO shifts preference toward LLM-generated content.

Unsupervised retrievers do not show a uniform bias; it varies by dataset.

Fine-tuning on LLM-generated corpora induces a strong pro-LLM bias.

Abstract

Dense retrieval is a promising approach for acquiring relevant context or world knowledge in open-domain natural language processing tasks and is now widely used in information retrieval applications. However, recent reports claim a broad preference for text generated by large language models (LLMs). This bias is called "source bias", and it has been hypothesized that lower perplexity contributes to this effect. In this study, we revisit this claim by conducting a controlled evaluation to trace the emergence of such preferences across training stages and data sources. Using parallel human- and LLM-generated counterparts of the SciFact and Natural Questions (NQ320K) datasets, we compare unsupervised checkpoints with models fine-tuned using in-domain human text, in-domain LLM-generated text, and MS MARCO. Our results show the following: 1) Unsupervised retrievers do not exhibit a uniform pro-LLM preference. The direction and magnitude depend on the dataset. 2) Across the settings tested, supervised fine-tuning on MS MARCO consistently shifts the rankings toward LLM-generated text. 3) In-domain fine-tuning produces dataset-specific and inconsistent shifts in preference. 4) Fine-tuning on LLM-generated corpora induces a pronounced pro-LLM bias. Finally, a retriever-centric perplexity probe involving the reattachment of a language modeling head to the fine-tuned dense retriever encoder indicates agreement with relevance near chance, thereby weakening the explanatory power of perplexity. Our study demonstrates that source bias is a training-induced phenomenon rather than an inherent property of dense retrievers.