Data, Not Model: Explaining Bias toward LLM Texts in Neural Retrievers

TL;DR

This paper investigates the source bias in neural retrievers favoring LLM-generated texts, revealing it originates from training data artifacts and proposing methods to mitigate this bias.

Contribution

The study identifies the root cause of source bias as dataset artifacts and introduces two effective strategies to reduce bias in neural retrievers.

Findings

Bias aligns with differences between human and LLM texts in embedding space.

Reducing artifact differences in training data decreases bias.

Adjusting LLM text vectors by removing bias projection also reduces bias.

Abstract

Recent studies show that neural retrievers often display source bias, favoring passages generated by LLMs over human-written ones, even when both are semantically similar. This bias has been considered an inherent flaw of retrievers, raising concerns about the fairness and reliability of modern information access systems. Our work challenges this view by showing that source bias stems from supervision in retrieval datasets rather than the models themselves. We found that non-semantic differences, like fluency and term specificity, exist between positive and negative documents, mirroring differences between LLM and human texts. In the embedding space, the bias direction from negatives to positives aligns with the direction from human-written to LLM-generated texts. We theoretically show that retrievers inevitably absorb the artifact imbalances in the training data during contrastive learning, which leads to their preferences over LLM texts. To mitigate the effect, we propose two approaches: 1) reducing artifact differences in training data and 2) adjusting LLM text vectors by removing their projection on the bias vector. Both methods substantially reduce source bias. We hope our study alleviates some concerns regarding LLM-generated texts in information access systems.