Exploring LLM biases to manipulate AI search overview

TL;DR

This paper investigates biases in LLM-based web search overview systems, demonstrating that these biases can be exploited through reinforcement learning to manipulate search result selections and highlighting safety concerns with such manipulations.

Contribution

It provides empirical evidence that biases in LLM overview systems can be exploited to influence search results, and analyzes safety risks associated with manipulation attacks.

Findings

Biases exist in LLM overview systems affecting source selection.

Reinforcement learning can optimize snippets to manipulate results.

Selection is driven by comparative advantages, not absolute quality.

Abstract

Modern large language models (LLMs) are used in many business applications in general, and specifically in web search systems and applications that generate overviews of search results - LLM Overview systems. Such systems are using an LLM to select most relevant sources from search results and generate an answer to the user's query. It is known from many studies that LLMs have different biases, in LLM Overview application both the source selection and answer generation stages may be affected by the biases of LLMs (here we are focusing mainly on the selection stage). This research is focused on investigating the presence of the biases in LLM Overview systems and on biases exploitation to manipulate LLM Overview results. Here we train a small language model using reinforcement learning to rewrite search snippets to increase their likelihood of being preferred by an LLM Overview. Our experimental setup intentionally restricts the policy to operate only on snippets and limits reward-hacking strategies, reflecting realistic constraints of web search environments. The results prove that LLM Overview systems have biases and that reinforcement learning in most of the cases can optimize snippet's content to manipulate LLM Overview results. We also prove that LLM Overview selections are driven by comparative rather than absolute advantages among candidate sources. In addition, we examine safety aspects of LLM Overview manipulation possibilities and show that context poisoning attacks can lead to inaccurate or harmful results.