Exploiting Positional Bias for Query-Agnostic Generative Content in Search

TL;DR

This paper reveals how positional bias in neural ranking models can be exploited to inject non-relevant content into search results, demonstrating a query-agnostic attack method and proposing mitigation strategies.

Contribution

It uncovers a positional bias vulnerability in neural ranking models and introduces a query-agnostic attack method using simulated promotional content, along with potential mitigation approaches.

Findings

Positional bias can be exploited to inject irrelevant content without affecting search ranking.

Lexical models are more resilient to content injection attacks than neural models.

Contextualising non-relevant content reduces negative impacts and may bypass filters.

Abstract

In recent years, neural ranking models (NRMs) have been shown to substantially outperform their lexical counterparts in text retrieval. In traditional search pipelines, a combination of features leads to well-defined behaviour. However, as neural approaches become increasingly prevalent as the final scoring component of engines or as standalone systems, their robustness to malicious text and, more generally, semantic perturbation needs to be better understood. We posit that the transformer attention mechanism can induce exploitable defects through positional bias in search models, leading to an attack that could generalise beyond a single query or topic. We demonstrate such defects by showing that non-relevant text--such as promotional content--can be easily injected into a document without adversely affecting its position in search results. Unlike previous gradient-based attacks, we demonstrate these biases in a query-agnostic fashion. In doing so, without the knowledge of topicality, we can still reduce the negative effects of non-relevant content injection by controlling injection position. Our experiments are conducted with simulated on-topic promotional text automatically generated by prompting LLMs with topical context from target documents. We find that contextualisation of a non-relevant text further reduces negative effects whilst likely circumventing existing content filtering mechanisms. In contrast, lexical models are found to be more resilient to such content injection attacks. We then investigate a simple yet effective compensation for the weaknesses of the NRMs in search, validating our hypotheses regarding transformer bias.