Adversarial Attacks against Neural Ranking Models via In-Context Learning

TL;DR

This paper introduces FSAP, a novel black-box adversarial attack framework using in-context learning of LLMs to generate misleading documents that can manipulate neural ranking models, highlighting a significant security threat.

Contribution

The paper presents FSAP, a new prompting-based attack method that does not require model access and can generate effective adversarial documents for neural ranking systems.

Findings

FSAP-generated documents outperform credible content in ranking.

Adversarial documents exhibit strong stance alignment and low detectability.

FSAP generalizes across different LLMs and query types.

Abstract

While neural ranking models (NRMs) have shown high effectiveness, they remain susceptible to adversarial manipulation. In this work, we introduce Few-Shot Adversarial Prompting (FSAP), a novel black-box attack framework that leverages the in-context learning capabilities of Large Language Models (LLMs) to generate high-ranking adversarial documents. Unlike previous approaches that rely on token-level perturbations or manual rewriting of existing documents, FSAP formulates adversarial attacks entirely through few-shot prompting, requiring no gradient access or internal model instrumentation. By conditioning the LLM on a small support set of previously observed harmful examples, FSAP synthesizes grammatically fluent and topically coherent documents that subtly embed false or misleading information and rank competitively against authentic content. We instantiate FSAP in two modes: FSAP-IntraQ, which leverages harmful examples from the same query to enhance topic fidelity, and FSAP-InterQ, which enables broader generalization by transferring adversarial patterns across unrelated queries. Our experiments on the TREC 2020 and 2021 Health Misinformation Tracks, using four diverse neural ranking models, reveal that FSAP-generated documents consistently outrank credible, factually accurate documents. Furthermore, our analysis demonstrates that these adversarial outputs exhibit strong stance alignment and low detectability, posing a realistic and scalable threat to neural retrieval systems. FSAP also effectively generalizes across both proprietary and open-source LLMs.