SearchAttack: Red-Teaming LLMs against Knowledge-to-Action Threats under Online Web Search

TL;DR

This paper introduces SearchAttack, a red-teaming method to evaluate the safety of search-augmented LLMs by evading safety measures and eliciting unsafe content, revealing vulnerabilities in current models.

Contribution

The paper presents a novel attack framework, SearchAttack, and a domain-specific illicit activity benchmark to assess and quantify risks in search-augmented LLMs.

Findings

SearchAttack effectively exposes vulnerabilities in search-augmented LLMs.

LLMs without web search can still produce harmful content due to inherent biases.

The framework enables comprehensive offline and online threat assessment.

Abstract

Recently, people have suffered from LLM hallucination and have become increasingly aware of the reliability gap of LLMs in open and knowledge-intensive tasks. As a result, they have increasingly turned to search-augmented LLMs to mitigate this issue. However, LLM-driven search also becomes an attractive target for misuse. Once the returned content directly contains targeted, ready-to-use harmful instructions or takeaways for users, it becomes difficult to withdraw or undo such exposure. To investigate LLMs' unsafe search behavior issues, we first propose \textbf{\textit{SearchAttack}} for red-teaming, which (1) rephrases harmful semantics via dense and benign knowledge to evade direct in-context decoding, thus eliciting unsafe information retrieval, (2) stress-tests LLMs' reward-chasing bias by steering them to synthesize unsafe retrieved content. We also curate an emergent, domain-specific illicit activity benchmark for search-based threat assessment, and introduce a fact-checking framework to ground and quantify harm in both offline and online attack settings. Extensive experiments are conducted to red-team the search-augmented LLMs for responsible vulnerability assessment. Empirically, SearchAttack demonstrates strong effectiveness in attacking these systems. We also find that LLMs without web search can still be steered into harmful content output due to their information-seeking stereotypical behaviors.