A Sentence Relation-Based Approach to Sanitizing Malicious Instructions

TL;DR

SONAR is a prompt sanitization framework that uses sentence relation metrics to detect and remove malicious instructions, significantly reducing attack success rates in LLM systems.

Contribution

Introduces SONAR, a novel sentence relation-based sanitization method that effectively defends against malicious instruction injections in LLMs.

Findings

SONAR reduces attack success rate to nearly zero.

Outperforms nine baseline defenses across multiple datasets.

Uses entailment and contradiction scores for sentence relation analysis.

Abstract

Retrieval-augmented generation and tool-integrated LLM agents increasingly depend on external textual sources. This reliance broadens the available attack surface, allowing adversaries to insert malicious instructions that trigger unintended model behaviors. Current defensive measures often utilize LLM-based detectors to filter such content, but these approaches remain vulnerable to optimization-based attacks. Additionally, training-based methods frequently fail to generalize to novel data distributions. To resolve these issues, we introduce SONAR, a prompt sanitization framework that identifies and removes injected content using metrics from natural language inference. Specifically, SONAR constructs a sentence-level relational graph across the user query and external data. By using entailment and contradiction scores as edge weights, the system identifies sentences that deviate from the core task. It then employs connectivity-driven pruning to eliminate flagged injection seeds and their related neighbors while maintaining benign context. Rigorous evaluations across several models and datasets show that SONAR reduces the attack success rate to nearly zero, significantly outperforming nine established baseline defenses.