Dynamics of Adversarial Attacks on Large Language Model-Based Search Engines

TL;DR

This paper models adversarial ranking attacks on LLM-based search engines as an infinitely repeated game, revealing complex dynamics and conditions that influence cooperation and attack strategies, with implications for system security.

Contribution

It introduces a game-theoretic framework to analyze the dynamics of ranking manipulation attacks on LLM search engines, providing new insights into defense strategies and system vulnerabilities.

Findings

Cooperation is more likely with forward-looking players.

Reducing attack success probability can paradoxically incentivize attacks.

Capping attack success rates may be ineffective in some scenarios.

Abstract

The increasing integration of Large Language Model (LLM) based search engines has transformed the landscape of information retrieval. However, these systems are vulnerable to adversarial attacks, especially ranking manipulation attacks, where attackers craft webpage content to manipulate the LLM's ranking and promote specific content, gaining an unfair advantage over competitors. In this paper, we study the dynamics of ranking manipulation attacks. We frame this problem as an Infinitely Repeated Prisoners' Dilemma, where multiple players strategically decide whether to cooperate or attack. We analyze the conditions under which cooperation can be sustained, identifying key factors such as attack costs, discount rates, attack success rates, and trigger strategies that influence player behavior. We identify tipping points in the system dynamics, demonstrating that cooperation is more likely to be sustained when players are forward-looking. However, from a defense perspective, we find that simply reducing attack success probabilities can, paradoxically, incentivize attacks under certain conditions. Furthermore, defensive measures to cap the upper bound of attack success rates may prove futile in some scenarios. These insights highlight the complexity of securing LLM-based systems. Our work provides a theoretical foundation and practical insights for understanding and mitigating their vulnerabilities, while emphasizing the importance of adaptive security strategies and thoughtful ecosystem design.