MindGuard: Intrinsic Decision Inspection for Securing LLM Agents Against Metadata Poisoning
Zhiqiang Wang, Haohua Du, Guanquan Shi, Junyang Zhang, HaoRan Cheng, Yunhao Yao, Kaiwen Guo, Xiang-Yang Li

TL;DR
MindGuard introduces a decision-level security mechanism for LLM agents that uses attention-based decision graphs to detect and attribute tool poisoning attacks with high accuracy and efficiency.
Contribution
It proposes the Decision Dependence Graph (DDG) for decision tracking and anomaly detection, a novel approach for securing LLM agents against metadata poisoning.
Findings
Achieves 94-99% precision in detecting poisoned invocations
Attains 95-100% attribution accuracy
Operates with processing times under one second
Abstract
The Model Context Protocol (MCP) is increasingly adopted to standardize the interaction between LLM agents and external tools. However, this trend introduces a new threat: Tool Poisoning Attacks (TPA), where tool metadata is poisoned to induce the agent to perform unauthorized operations. Existing defenses that primarily focus on behavior-level analysis are fundamentally ineffective against TPA, as poisoned tools need not be executed, leaving no behavioral trace to monitor. Thus, we propose MindGuard, a decision-level guardrail for LLM agents, providing provenance tracking of call decisions, policy-agnostic detection, and poisoning source attribution against TPA. While fully explaining LLM decision remains challenging, our empirical findings uncover a strong correlation between LLM attention mechanisms and tool invocation decisions. Therefore, we choose attention as an empirical…
| Method | Track | Detect | Attribute | Plug-in | Model-free |
|---|---|---|---|---|---|
| Static Scan∗ | ✗ | ✗ | ✓ | ✓ | ✗ |
| Architect Isolate∗ | ✗ | ✓ | ✗ | ✗ | ✗ |
| Behavior Audit | ✗ | ✓ | ✗ | ✓ | ✗ |
| Policy Enhance | ✗ | ✓ | ✗ | ✗ | ✓ |
| MindGuard† | ✓ | ✓ | ✓ | ✓ | ✓ |
| Metric | Model | ToolACE | ToolAlpha | ToolBench | MCPTox | ||||
|---|---|---|---|---|---|---|---|---|---|
| Qwen3-8b |
|
|
|
|
|||||
| Qwen3-14b |
|
|
|
|
|||||
| Gemma-9b |
|
|
|
|
|||||
| Qwen3-8b |
|
|
|
|
|||||
| Qwen3-14b |
|
|
|
|
|||||
| Gemma-9b |
|
|
|
|
| Vertex | Logical Concept | Content |
|---|---|---|
| User Query | user query | |
| Available Tools | each tool’s metadata | |
| Invoked Tool Name | generated pre-argument invocation block | |
| Invoked Arguments | generated invocation argument value. |
| LLM | MCPTox (#) | InjecAgent (#) | RAS-Eval (#) | |||
|---|---|---|---|---|---|---|
| Normal | Poisoned | Normal | Poisoned | Normal | Poisoned | |
| Qwen3-8b | 744 | 68 | 456 | 12 | 919 | 6 |
| Qwen3-8b† | 606 | 289 | 219 | 184 | 605 | 182 |
| Qwen3-14b | 755 | 64 | 487 | 10 | 825 | 0 |
| Qwen3-14b† | 580 | 343 | 232 | 187 | 96 | 687 |
| Phi-4† | 204 | 676 | 5 | 61 | 125 | 0 |
| Mistral-7b | 571 | 87 | 477 | 23 | 841 | 0 |
| Gemma2-9b | 858 | 187 | 486 | 4 | 829 | 0 |
| Dataset | Task | Metric (Clean) | Qwen3 | Phi | Mistral | Gemma2 | Total∗ | |||
|---|---|---|---|---|---|---|---|---|---|---|
| Qwen3-8b | Qwen3-8b† | Qwen3-14b | Qwen3-14b† | Phi-4† | Mistral-7b | Gemma2-9b | ||||
| MCPTox [9] | Detection | 98.8 (99.1) | 96.7 (97.3) | 98.0 (98.0) | 89.0 (95.7) | 93.5 (94.9) | 94.6 (96.7) | 95.4 (99.0) | 97.7 | |
| 93.4 (99.4) | 98.1 (99.2) | 87.1 (97.0) | 92.9 (97.9) | 96.8 (98.2) | 81.6 (94.4) | 94.7 (99.2) | 96.3 | |||
| 99.1 (99.9) | 98.7 (99.5) | 96.3 (99.5) | 95.0 (98.5) | 90.8 (97.1) | 94.7 (98.3) | 98.1 (99.6) | 98.5 | |||
| Attribution | 100 (100) | 97.0 (97.0) | 100 (100) | 97.5 (97.5) | 95.5 (95.5) | 100 (100) | 100 (100) | 97.8 | ||
| InjecAgent [50] | Detection | 100 (100) | 99.8 (100) | 99.6 (98.0) | 97.0 (95.7) | 97.0 (94.9) | 99.8 (100) | 100 (99.0) | 99.9 | |
| 100 (100) | 99.9 (99.9) | 97.3 (98.7) | 99.7 (99.8) | 84.3 (97.1) | 97.6 (98.6) | 100 (100) | 98.9 | |||
| 100 (100) | 99.9 (99.9) | 99.9 (100) | 99.6 (99.7) | 97.4 (98.1) | 99.7 (99.9) | 100 (100) | 99.7 | |||
| Attribution | 100 (100) | 100 (100) | 100 (100) | 99.6 (99.6) | 100 (100) | 100 (100) | 100 (100) | 99.8 | ||
| RAS-Eval [51] | Detection | 99.9 (100) | 98.7 (99.5) | – | 99.0 (99.7) | 93.5 (98.6) | – | – | 98.1 | |
| 85.6 (98.7) | 98.5 (99.4) | – | 98.5 (99.9) | 96.8 (98.9) | – | – | 97.7 | |||
| 95.3 (99.4) | 99.1 (99.7) | – | 99.7 (99.9) | 90.8 (98.1) | – | – | 99.0 | |||
| Attribution | 100 (100) | 97.1 (97.1) | – | 100 (100) | 97.5 (97.5) | – | – | 97.9 | ||
| =0.3 | =0.5 | =0.7 | =0.9 | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| TPR | FPR | FPR(Clean∗) | TPR | FPR | FPR(Clean∗) | TPR | FPR | FPR(Clean∗) | TPR | FPR | FPR(Clean∗) | |
| CFI Check | 93.1 | 17.6 | 3.5 | 75.3 | 2.9 | 0.1 | 44.6 | 0.3 | 0.0 | 25.7 | 0.0 | 0.0 |
| DFI Check | 98.5 | 30.9 | 8.5 | 95.1 | 7.4 | 1.9 | 92.1 | 1.6 | 0.4 | 87.5 | 0.5 | 0.3 |
| Mixed | 98.6 | 30.0 | 8.2 | 95.1 | 7.1 | 1.9 | 91.9 | 1.2 | 0.6 | 87.4 | 0.5 | 0.4 |
| Paradigm | Method | Qwen3-8b† | Qwen3-8b | Phi-4† | |||
| TPRFPR | TPRFPR | TPRFPR | |||||
| Atten. Track | 62.6 48.3 | 26.5 38.4 | 69.9 58.8 | ||||
| Static Scan | LLM-Guard | 33.2 36.1 | 36.8 35.3 | 35.2 39.7 | |||
| LLM Detect | 42.2 38.8 | 50.0 41.0 | 40.1 45.1 | ||||
| Architect Isolate | CaMeL | 00.0 0.00 | 00.0 0.00 | 00.0 0.00 | |||
| PFI | 00.0 0.00 | 00.0 0.00 | 00.0 0.00 | ||||
| Behavior Audit | MELON | 14.3 4.00 | 47.1 4.50 | 0 2.0 0.50 | |||
| MCIP | 58.5 54.0 | 57.4 51.8 | 56.4 54.9 | ||||
\rowcolorgray!20[][0]
|
Ours | 96.5 5.20 | 94.1 3.90 | 91.0 9.5 | |||
| Paradigm | Method | Qwen3-8b† | Qwen3-8b | Phi-4† | |||
| P R | P R | P R | |||||
| Atten. Track | 38.2 62.6 | 5.9 26.5 | 79.3 69.9 | ||||
| Static Scan | LLM-Guard | 30.5 33.2 | 8.7 36.8 | 74.6 35.2 | |||
| LLM Detect | 34.2 42.2 | 10.0 50.0 | 74.7 40.1 | ||||
| Architect Isolate | CaMeL | 00.0 0.00 | 00.0 0.00 | 00.0 0.00 | |||
| PFI | 00.0 0.00 | 00.0 0.00 | 00.0 0.00 | ||||
| Behavior Audit | MELON | 7.4 14.3 | 8.4 47.1 | 93.0 2.0 | |||
| MCIP | 34.1 58.5 | 9.2 57.4 | 77.3 56.4 | ||||
\rowcolorgray!20[][0]
|
Ours | 97.1 92.60 | 94.0 91.20 | 91.0 93.0 | |||
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Malware Detection Techniques · Security and Verification in Computing · Network Security and Intrusion Detection
MindGuard: Intrinsic Decision Inspection for Securing LLM Agents
Against Metadata Poisoning
Zhiqiang Wang,Haohua Du*,Guanquan Shi,Junyang Zhang,Haoran Cheng,Yunhao Yao,Kaiwen Guo,Xiang-Yang Li
Abstract
The vulnerability of LLM decision-making to pollution from untrusted external tool data injection poses a critical security threat to LLM agents. Prevailing defenses focus on isolating trusted decision contexts (pre-decision) or monitoring invocation behaviors (post-decision), failing to inspect the actual point where such injections take effect, i.e., the LLM’s internal decision logic. These approaches are therefore fundamentally inadequate, especially ineffective against metadata-based injection (i.e., metadata poisoning), which pollutes the decision context via indispensable tool metadata and generates no explicit and observable malicious invocations.
To bridge this gap, we propose Decision Inspection, a novel security paradigm that scrutinizes the LLM’s internal tool-call logic by tracking and verifying decision provenance. We introduce the Decision Dependency Graph (DDG) to characterize the LLM’s decision-making logic as the dependency of call decisions on different input context elements (e.g., query, metadata). We then innovatively design an attention-based mechanism to quantify these decision dependencies. Built upon the DDG, we formally define two anomalies to enable immediate, anomaly-based detection of poisoned calls without relying on predefined security rules. We implement this paradigm as MindGuard, providing the first decision-level poisoned call detection and dynamic, runtime attribution of attack sources. Evaluations across multiple real-world datasets and LLM demonstrate that MindGuard outperforms SOTA methods, achieving the highest detection rate with the lowest false alarms (Average Precision97.6%) and near-perfect attribution accuracy (98.6%). Crucially, MindGuard is a self-inspection plugin that requires no external models, saving 1,000+ tokens per invocation and incurring subsecond-level runtime costs.
1 Introduction
The integration of Large Language Models (LLMs) with external tools has catalyzed a paradigm shift from passive text generators to proactive agent systems [1, 2, 3]. These agent systems leverage LLMs to generate tool-call decisions by synthesizing user queries, tool metadata (e.g., tool descriptions, configuration files, etc.), and execution histories, thereby autonomously invoking tools to execute complex tasks. While powerful, this architectural evolution introduces unprecedented security vulnerabilities. For example, Google Gemini Assist’s browsing tool was exploited to exfiltrate private user data [4]; malicious npm packages compromised AI agents to steal cryptocurrency wallets and credentials, affecting over 1,400 users [5]; Slack AI was manipulated to expose all user inputs from private channels [6].
These vulnerabilities resurrect a classic Code Injection [7] flaw, but in a form uniquely native to the LLM agent paradigm. In traditional security models, this vulnerability class, exemplified by SQL Injection [8], is mitigated by trusting only code, not data. Systems enforce this separation through strict, well-defined interfaces and mechanisms like parameterization, ensuring that untrusted user data is never mistakenly executed as code. However, this entire defense model collapses in LLM agents. In LLM agents, natural language serves as the unified communication channel, inherently intermingling trusted code (e.g., user instructions) with untrusted data (e.g., third-party tool metadata). LLMs perform indiscriminate reasoning across this mixed context, fundamentally blurring trust boundaries, leading to two critical failures: 1) Code/Data Confusion: The LLM’s reasoning treats untrusted data as executable implicit code111Implicit code refers to data that exists in passive forms (e.g., metadata, configure file), yet during the semantic understanding and decision process of LLMs, it is interpreted as active call instructions (code). These instructions play a pivotal role in driving and manipulating the core semantic sequences that guide the model’s call decision-making. 2) Privilege Amplification: The agent is granted an omnipotent privilege pool encompassing all tools, enabling such confusions to escalate into cross-tool privilege escalation.
Metadata Poisoning [9, 10, 11, 12, 13], the focus of this paper, is a stealthy exploitation of this injection (Fig 1). It begins during the Registration Stage (Pre-decision). The attack payload is tightly embedded within the tool’s natural language description and loaded into the agent’s context, where it lies dormant as passive data During Planning Stage (Decision-making), this payload is dynamically triggered by specific user queries and activated as implicit code, hijacking the tool-call decision. Ultimately, the invocation remains legitimate in both privilege and syntactic structure upon Execution Stage (Post-decision). Metadata poisoning constructs a cross-stage implicit delegation chain that leverages legitimate tool invocations to perform malicious operations, where the malicious intent only transiently manifests during the decision-making process.
Research Gap. Current defenses centered on Pre-decision Purification and Post-decision Auditing. 1) Pre-decision Purification cannot access the dynamic runtime context to adjudicate. Static scanning [14, 15, 16] sanitizes the metadata to remove descriptions that could trigger malicious behavior. However, the complexity of natural language prevents it from reliably distinguishing maliciously crafted content, which only triggers as implicit code in specific runtime contexts, from benign data. This fundamental limitation persists even when considering advanced LLM-based detection methods (Table II). Architectural isolation [17, 18, 19], which employs a radical strategy of isolating all execution results for planning, fails because the metadata itself is indispensable for tool use [18]. 2) Post-decision Auditing cannot verify the intent provenance of a call decision. Post-decision Auditing primarily focuses on anomaly behavior recognition [20, 21] or security policy enforcement [22, 23] through monitoring observable malicious call sequences. However, they focus predominantly on what happened rather than why it happened, thus remaining unable to determine whether a legitimate-looking call (e.g., Send(to=[email protected] )) genuinely originated from the user’s instruction (send email to Alice) or was already influenced by malicious metadata. We argue that a robust defense must address the fundamental vulnerability inherent in using natural language as a unified communication channel: the indistinguishability between implicit code and data at the pre-decision stage, and the unverifiability of dynamic invocation intent at the post-decision stage.
Our Solution. To bridge this gap, we propose Decision Inspection, the first decision integrity verification mechanism that directly scrutinizes the LLM’s internal decision logic, precisely where malice manifests. Decision inspection bypasses the intractable problem of semantically distinguishing code from data (pre-decision) or verifying invocation intent (post-decision) within natural language. Instead, it directly tracks dynamic decision provenance, i.e., which contextual elements influenced the tool-call decision (identified as implicit code) and verifies the integrity of this provenance to safeguard user intent.
Challenges. Building such an effective decision inspection framework presents two key challenges:
C1: Robust Decision Provenance Tracking. Given the intrinsically opaque nature of LLMs, direct causal explanation of LLM decision-making remains intractable. Therefore, we must identify a stable proxy signal that can reliably track and quantify decision provenance to identify the specific contextual data that influences the call decision, i.e., transforming into implicit code during the LLM’s reasoning.
C2: Dynamic Decision Integrity Checking. The diverse nature of malicious intents makes statically predefined security rules insufficient. We therefore require dynamic decision integrity verification that operates without relying on such rules to prevent decision hijacking.
MindGuard. Overcoming challenges posed by decision inspection requires multiple technical advances. First, we introduce the Decision Dependency Graph (DDG) as a formal model to characterize the LLM’s decision logic. Then, we propose using attention activation as a practical signal for tracking decision provenance to construct DDG, designing a two-stage sink filtering and Total Attention Energy computation to quantify dependencies from noisy attention. While theoretically proof remains challenging, we validate its statistical robustness over 12,000+ tool-call samples (Addressing C1). Building upon the DDG, we then develop an anomaly-based decision integrity verification mechanism. Instead of relying on predefined security rules, it detects attacks and traces the poisoned context by leveraging two anomalies that serve as evidence of an integrity violation (Addressing C2). We implement our prototype system as MindGuard, an intrinsic self-inspection plug-in for existing frameworks. MindGuard is the first framework for verifying decision integrity during the LLM’s reasoning process, requiring no external models and incurring zero additional token overhead. Table I compares MindGuard with existing security mechanisms.
Contributions. We make the following contributions:
Threat Conceptualization. We formulate LLM-style code injection, exemplified by metadata poisoning, as a decision integrity violation, shifting the focus from input/output to the integrity of the LLM’s internal decision logic.
Decision Inspection Paradigm. We introduce the Decision Dependency Graph (DDG), the first formal model to characterize LLM’s decision logic. We design a robust attention-based mechanism to track decision provenance and propose an anomaly-driven integrity verification method without relying on predefined security rules.
Practical Guardrail. We implement MindGuard, the first self-inspection system for decision integrity Verification. MindGuard constructs and verifies the DDG during call generation, enabling immediate detection of poisoned calls and precise attribution to malicious metadata. It achieves over 97% accuracy while operating with zero additional token overhead and requiring no auxiliary models.
2 Preliminary and Background
2.1 Tool-Augmented LLM Agents
In practice, the operational workflow of a Tool-augmented LLM agent consists of three core phases (Fig. 1):
Stage 1: Registration (Pre-decision). Based on task requirements, the user selects and registers a set of tools, denoted as . The agent then automatically loads the corresponding metadata into the LLM’s context, establishing the foundational knowledge for all subsequent decision-making.
Stage 2: Planning (Decision-making). The agent’s active task execution is triggered by a high-level user query , which defines the overall objective. Complex tasks often necessitate multi-step planning. At each step , the agent constructs a decision context by amalgamating the user query , the registered tool metadata , and optionally includes the interaction history . The agent’s core LLM, acting as a decision function , processes this context to generate the next tool-call decision:
[TABLE]
Stage 3: Execution (Post-decision). The tool call is invoked and executed by the corresponding tool with parameters . The execution result is observed and used to update the interaction from to . This loop of Planning followed by Execution continues iteratively until the user’s objective is fulfilled or a terminal condition is met.
The entire lifecycle establishes a delegation chain: the user delegates authority to the agent, which, through LLM-based reasoning, delegates operations to tools for execution.
2.2 Metadata Poisoning
The decision context of an LLM agent inevitably commingles high-trust user queries with low-trust third-party tool metadata , creating a natural attack surface for Code Injection (i.e., Untrusted data is introduced and executed as code) [7]. We term this attack as metadata poisoning, where adversaries embed attack payloads within tool metadata during registration of seemingly benign tools, subsequently manipulating the LLM planning to invoke other legitimate tools for executing malicious operations (Fig. 1). Three key properties make metadata poisoning particularly challenging to defend: 1)Attack payloads are tightly coupled with tool’s natural language description 2) Attack are injected at the registration stage and remain persistently dormant as passive data, only influencing the model’s decisions when activated as implicit code by a specific query; 3) The malicious tool requires no explicit invocation, leaving no observable traces for auditing.
A Broad Class of Real Attacks. Metadata Poisoning refers not to a single specific attack, but to a class of attacks where attack payloads are injected into tool metadata during the Registration Stage and dynamically triggered during the Planning Stage. Specially, it includes Tool Poisoning Attack [9, 24, 10], Puppet Attack [11], Rug Pull Attack [11, 12], Supply Chain Attacks [5],Tool Invocation Prompt Attack [25], Configuration File Poisoning [26], etc.
2.3 Existing Defense Paradigm
Existing defense can be classified into two categories
Pre-decision Purification aims to build a trusted environment before LLM decision-making, primarily through two mechanisms: Static scanning [14, 15, 16] employs syntactic or semantic analysis to scan for and filter potentially malicious data. However, when both code and data are expressed in natural language, these methods fail to semantically disentangle benign metadata from obfuscated malicious instructions. Architectural Isolation [17, 18, 19] assumes metadata is trustworthy for planning and sandboxes all tool execution results for each decision step. This approach is fundamentally circumvented by metadata poisoning, as the attacker directly pollutes the trusted planning context itself. As evidenced in Table II, the purification rates of different SOTA methods against metadata poisoning are consistently below 41%. More importantly, these pre-decision purification methods lack runtime context and thus cannot identify the dynamic malicious intent that emerges from the LLM’s decision logic.
Post-decision Auditing primarily focuses on anomaly behavior recognition [20, 21] or security policy enforcement [22, 23] through monitoring observable invocation and call sequences. Although they can monitor observable malicious calls, these methods focus solely on what occurs rather than why it occurs. Consequently, they are ineffective against calls that appear legitimate but whose decision logic has been contaminated by malicious metadata. As illustrated in Fig. 1, the entire observable call chain appears legitimate: User is authorized to send an email to [email protected], and execution result 3:00pm of third-party tool Time exhibits no malicious operations. Furthermore, they cannot verify semantic legitimacy at the level of intent, for instance, confirming whether [email protected] is truly Alice’s email address or an attacker-controlled account.
Ultimately, existing defenses operate at the input () or output () level, while the true malice resides in the LLM’s internal decision logic (i.e., Decision Hijack). We argue, therefore, that security must shift to the inspection of ’s internal execution to ensure its integrity.
3 Threat Model
3.1 System and Trust Model
We consider a standard tool-augmented agent system (as illustrated in Figure 1) composed of three primary entities:
User (): A human operator who issues high-level queries to the agent. The user is considered trusted, and their query represents the high-trust source of intent.
Agent (): The LLM-powered entity, which acts as a decision-making function . The agent itself is a high-privilege entity, authorized to orchestrate and delegate operations across all provisioned tools. We assume the agent’s logic is benign but gullible, i.e., it indiscriminately reasons over all data in its context, creating the Data/Code Confusion vulnerability.
Third-party Tools (): Collection of tools available to the agent. These tools, often sourced from open, unregulated marketplaces (e.g., MCP.so[28] and SmitheryAI [29]), are considered untrusted. They provide two types of low-trust input to the agent’s context:
- •
Metadata (): Static descriptions (e.g., tool description) loaded during the Registeration phase.
- •
Execution Results (): Dynamic outputs generated from tool invocations.
Our Scope. We focuses on the threat from untrusted metadata . We consider this the more stealthy and persistent vector, as existing architectural isolation (e.g., plan-then-execute) can partially mitigate the risk from untrusted execution results , but cannot, by definition, isolate the metadata that the LLM requires for its reasoning (Table II).
3.2 Attack Model
Attacker Goals - Implicit Delegation. The attacker’s ultimate objective is to induce the agent to misuse its legitimate authority to execute malicious operations. These attacks are not achieved by directly invoking a malicious tool, but by subverting the agent’s internal decision logic, effectively transforming it into a confused deputy that unknowingly performs the attacker’s bidding. As illustrated in Figure 1, the malicious operation is executed via implicit delegation chains (T_{M}$$\rightarrowSend) concealed within legitimate workflow sequences (User TimeSend), rendering them undetectable through execution monitoring.
Attacker Capability. Attackers can achieve cross-tool privilege escalation simply by registering a seemingly legitimate toolset (e.g., Calendar) that conceals a malicious tool , without requiring any modification to the high-privilege tool set (e.g., Email) itself. This requires only minimal capabilities: 1) publishing tools with malicious metadata disguised as benign functionality to public markets; 2) persuading a user to select and register this tool.
Attack Mechanism. As illustrated in Fig. 1, attackers construct implicit delegation chains through the following process: 1) Tool Release: The attacker publishes a seemingly normal Calendar toolset that covertly includes a malicious tool containing a poisoned metadata payload to public tool marketplaces like MCP.so [28] or SmitheryAI [29]. 2) Context Inject (poisoned metadata dormant as static data): Once the user selects the Calendar toolset, the agent automatically loads the metadata into the decision context. The poisoned payload remains dormant as seemingly benign passive data, making it notoriously difficult to detect through static analysis. However, it has already compromised the LLM’s decision context, lying in wait to be activated by specific runtime cues. **3) Decision Hijack
(poisoned metadata activated as implicit code):** This latent threat activates when triggered by a specific user intent (e.g., sending an email). Within this specific context, the dormant data is dynamically activated as implicit code, hijacking the LLM’s decision-making process. This process constitutes a violation of decision integrity (§3.3). 4) Legitimate Invoke. Once the tool-call decision is made, the generated invocation appears outwardly legitimate and valid (e.g., Send(to=[email protected] )). The agent executes this seemingly legitimate call, fulfilling the attacker’s malicious objectives. Thus, the malice is a transient event, occurring exclusively during the LLM’s decision process.
Practical Feasibility. The proliferation of open tool protocols like MCP [30] has enabled attackers to easily distribute malicious tools through public marketplaces. Studies confirm this threat’s viability, showing malicious tools are readily published and perceived as trustworthy [11, 31, 32], with over 75% of users selecting malicious servers in one study [11]. Once adopted, attackers can silently modify tool metadata without triggering alerts (e.g., Rug Pull Attack [11, 12]), ensuring persistent evasion.
3.3 Formulation: Decision Integrity Violations
The attack actually takes effect within the LLM’s decision function , which can be formalized as:
[TABLE]
where is the metadata context poisoned by , and () represents the maliciously generated tool call and its parameters. This decision process is directly analogous to the intern execution logic of a traditional program. Consequently, we formally model the core vulnerability of metadata poisoning attacks as a violation of decision integrity to ground our defense in established security principles. We identify two specific manifestations:
Control-Flow Integrity (CFI) Violation.
1) Classical Definition: Ensures a program’s execution follows a predetermined path, impervious to hijacking by untrusted data. 2) Agent Manifestation: The poisoned metadata , acting as implicit code, illegitimately influences the decision on which tool to invoke (). This subverts the execution flow that should be solely determined by the user’s intent .
Data-Flow Integrity (DFI) Violation.
1) Classical Definition: Ensures that a program’s internal data values are not corrupted by untrusted sources. 2) Agent Manifestation: The poisoned metadata pollutes the decision logic leading to the invocation arguments of a legitimate cal ().
Therefore, the defense problem is re-framed as the need to perform CFI and DFI checks on the execution logic of the decision function itself. A robust mechanism must answer the fundamental question: Is the decision logic of invoked tools (CFI Check) or the argument values (DFI Check) influenced by untrusted metadata?
3.4 Decision Inspection Defense
This formulation leads us to propose Decision Inspection, a defense paradigm that shifts the security boundary to the LLM’s internal decision-making process. Decision Inspection dynamically scrutinizes the integrity of LLM decisions by tracing their contextual provenance, and performs integrity at the decision level
Defense Goals. Specifically, the decision inspection defense consists of the following three goals:
1) Context Provenance Disentanglement (Tracking): Formally model and quantify the dependency influence () of all input context elements on the final generated tool call.
2) Decision Integrity Verification (Detection) Inspect the provenance () to verify its integrity. A poisoned call is defined as an integrity violation where the call decision is driven by untrusted implicit code rather than user intent.
3) Poisoned Source Purification (Attribution) Upon detection, immediately identify the source of the poisoning to sanitize the agent’s context by removing the malicious metadata to prevent future violations.
Defender Capabilities. We assume that the defender possesses white-box access to the LLM’s inference process, allowing visibility into the attention matrices, input contexts, and the final output tool call, without necessitating any modification to the model itself. This setup is highly suitable for two primary practical scenarios: 1) for model service providers (e.g., Anthropic, OpenAI) integrating this defense mechanism as a value-added security feature, just like cloud providers today offer built-in DDoS protection; 2) for enterprises deploying open-source LLMs for enhanced internal auditing and protection.
4 Methodology
To achieve decision inspection, we first introduce the Decision Dependency Graph, a formal model that characterizes the LLM’s decision logic (§4.1). We then devise an attention-based mechanism to track the influence of different contextual elements on decisions (§4.2). Finally, we propose an integrity-anomaly-based verification scheme to accurately identify compromised decisions (§4.3).
4.1 Characterization: Decision Dependency Graph
To facilitate the three objectives of decision inspection, we require a formal model to characterize the internal decision logic of , analogous to how Program Dependence Graphs [33] (PDGs) model the internal logic flows of deterministic programs in traditional security. However, such deterministic models are ill-suited for the probabilistic nature of LLM decision-making, which operates not through fixed control flows but through dynamic, context-dependent semantic reasoning. To bridge this gap, we propose the Decision Dependency Graph (DDG), a formal representation adapted to the probabilistic, context-sensitive decision environment of LLMs. We formally define the Decision Dependency Graph as a weighted, directed graph, , that is dynamically constructed for each tool call decision. Its vertices represent logical concepts (User Query , Tools Metadata , Call Decision ). Crucially, to operationalize our CFI/DFI goals, we decompose the decision vertex into two components:
- •
(Invoked Tool Name), representing the Control-Flow checkpoint, answers the question: who controlled the choice of the tool being called?
- •
(Invocation Arguments), representing the Data-Flow checkpoint, answers the question: from where did the parameters’ data originate?
The edges represent the dependency influence from an input vertex ( ) to an output decision vertex (). The weights quantify the strength of this influence. The DDG thus transforms the abstract challenge of verifying decision integrity into a concrete, measurable graph representation, where CFI/DFI violations can be checked by analyzing the edge weights.
4.2 Construction: Decision Dependency Tracking
Constructing the DDG requires a mechanism to quantify the dependency influence weights . This presents a fundamental challenge due to the opacity of LLM. We propose using attention as a practical proxy signal for decision quantification and introduce a series of purification and aggregation mechanisms to compute robust dependency influence weights (§5.2). Importantly, we do not attempt to position attention as a causal explanation for the entire decision process. Instead, we leverage it as a practical correlation signal, grounded in the strong statistical relationship between decision sources and attention activation. This correlation is sufficient to perform decision provenance tracking, determining exactly which contextual elements (implicit code) influenced the final tool call.
Intuition and Observation. The attention mechanism serves as a dynamic, context-aware retrieval operation at each decoding step. Thus, a higher attention weight implies an input token was deemed more salient, allowing its semantic features to be more heavily injected into the output being formed. Intuitively, this mechanism may provide a window into the model’s decision process: contexts exerting greater influence on a tool-call decision should correspondingly attract higher attention during its generation. Our empirical observations of attention patterns corroborate this relationship, i.e, Context elements that influence call decisions exhibit pronounced attention activation (Fig. 2): In a Normal Call, the decision originates from the user query, resulting in strong activation for Query. Conversely, in a Poisoned Call, the decision is hijacked by poisoned tool Common metadata and deviates from the user query, causing Common to exhibit strong activation while User activation weakens. Additionally, since the final tool call must rely on the invoked tool’s own metadata for generation, the invoked tool’s metadata remains one of the decision sources and maintains high activation (ReadFile in poisoned call and ListDirectory in normal call).
Based on this, we propose using Total Attention Energy (TAE), a squared-sum of attention activations, as a robust metric to quantify this influence (). The squaring (energy) operation enhances the signal from high-influence tokens (code execution) while minimizing the noise from low-influence tokens (passive data):
[TABLE]
where denotes the token sequence of a vertex, is the attention matrix, and represents our proposed two-stage sink filter that processes raw attention weights to filter the attention sink noise [34, 35]. The detailed implementation steps are described in §5.2.
Statistical Verification. To rigorously evaluate the effectiveness of TAE, we conduct a statistical validation assessing its capability along two key dimensions: 1) Distinguishability, the ability to differentiate contexts that influence decisions from those that do not; and 2) Correlation, the capacity to quantitatively reflect the degree of influence. Our validation dataset comprises 12,000+ heterogeneous clean tool call samples from various benchmarks. In these clean samples devoid of malicious instructions, we can establish ground-truth labels by designating the invoked tool’s metadata and user query as decision sources (positive class), while treating other tool metadata as non-sources (negative class). Contextual elements serving as the decision source (Invoked Tool and Query) exhibit significantly higher TAE attention score compared to non-source elements (Fig. 3(a)). Furthermore, we employ two statistical metrics to valid TAE: 1) Common Language Effect Size (CLES) [36] to assess its capacity to distinguish between decision sources and non-sources (Distinguishability); and 2) point-biserial correlation () [37] to measure the strength of association between TAE values and ground-truth provenance labels (Correlation). Our validation (Table III) confirms this hypothesis. The exceptional CLES values (98.3%) and strong correlations (82.11%) validate TAE as a reliable signal for decision provenance, constituting a footprint for implicit code that is statistically robust and highly distinguishable from the footprint of passive data (non-sources).
4.3 Utilization: Decision Integrity Verification
Having established the Decision Dependency Graph as our formal model (§4.1) and the attention-based mechanism to instantiate it (§4.2), we now operationalize our core defense, i.e., Decision Integrity Verification. We propose an anomaly-driven decision integrity verification mechanism that detects security violations by identifying two forensic evidence for a decision hijack in the DDG.
Normal Call (Integrity Compliant). Normal calls, which satisfy decision integrity, derive their decision provenance overwhelmingly from two legitimate contexts: 1) User Query provides the fundamental task intent; 2) The Invoked Tool supplies the essential reference required to generate its specific call (Fig 3 (a)).
Poisoned Call (Integrity Violation). Poisoned calls that violate decision integrity manifest when malicious tool metadata hijacks the LLM’s decision logic (Cause of Hijack), deviating it from the user’s intent (Consequence of Hijack). This violation is captured in the DDG by two forensic anomalies:
Implicit Delegation Anomaly - Cause of Hijack: We observe a high-weight edge from an untrusted, non-invoked tool’s metadata to a decision vertex ( or ). This is the direct evidence of a CFI/DFI breach and is formally captured by the Implicit Delegation Anomaly:
Definition 1** (Implicit Delegation Anomaly).**
Let be the user query source vertex and be the input context vertex corresponding to the invoked tool . There exists an implicit delegation anomaly if:
[TABLE]
where is an empirical threshold for high-weight edges, indicating unexpected influence from contexts beyond legitimate sources.
User Intent Dilution Anomaly - Consequence of Hijack: As a necessary consequence of this violation, the influence from the trusted () is suppressed. This symptom is formally defined as the User Intent Dilution Anomaly:
Definition 2** (User Intent Dilution Anomaly).**
Let be the user query source vertex and be the invoked tool target vertex. User intent dilution occurs if:
[TABLE]
where is an empirical threshold, indicating that the user query’s influence is significantly below expected levels.
Fig. 3 (b) statistically demonstrates the presence of the two integrity anomalies described above. While Definitions 1 and 2 formalize the evidence, tuning absolute thresholds () is brittle and not practical or robust (especially when processing contexts containing varying numbers of tools. As the tool count increases, the TAE activation values become constrained due to normalization effects). We therefore propose a single, relative metric, the Decision Integrity Ratio (DIR), to quantify the integrity violation. It normalizes the anomalous signal from an uninvoked tool against the combined influence of the legitimate source: user’s legitimate intent and the invoked tool’s own description, simplifying the detection logic to a single, more stable hyperparameter. Detailed implementation is in §5.3
5 System Design
We implement MindGuard (Fig. 4), a prototype guardrail that performs intrinsic self-inspection, constructs and analyzes the Decision Dependency Graph during the reasoning process to achieve our security goals. This section introduces the design of Context Vertex Parser (§5.1), Dependency Graph Builder (§5.2), and Decision Integrity Defender (§5.3) in MindGuard.
5.1 Context Vertex Parser
The Context Vertex Parser aims to instantiate the formal DDG model defined in §4.1. It analyzes the LLM’s input context () and output call () to extract the graph’s vertices and map them to corresponding token spans.
Context Parsing. The parser first identifies the logical concepts that constitute the DDG vertices (). Critically, to operationalize the CFI and DFI checks that form the basis of our methodology (as established in §4.1), the parser can further decompose the output decision vertex into its two core integrity checkpoints: 1) Invoked Tool Name (): representing the CFI checkpoint; 2) Invocation Arguments (): representing the DFI checkpoint.
Position Mapping. For each vertex, the parser performs position mapping to identify its exact token span in the attention matrix. This mapping (i.e., ) is crucial for the Dependency Graph Builder (§4.2) to quantify influence.
We treat the and vertices differently to capture their roles in CFI/DFI precisely. For , we track its semantic provenance to understand the reasoning behind the tool choice. We define its token span as the pre-argument-invocation block, which serves as a summary of the model’s reasoning (inspired by research on chain-of-thought [38, 39] ). For , we focus on data origin. Argument values are often copied directly from the context. Therefore, we define the token spans of these vertices as the generated argument values. Table IV details the specific token content corresponding to each logical vertex. Our localization process first identifies token content for each vertex at the string level through pattern matching, which is then converted into its final token spans of indices.
5.2 Dependency Graph Builder
The Dependency Graph Builder quantifies the influence weights () for DDG edges, transforming raw attention matrices into robust influence dependencies as defined in our methodology. This process (Algorithm 1) comprises four main steps: Cross-Layer Attention Combination, Sink Filter, Attention Partition, and TAE Aggregation, designed to purify and aggregate the dependency evidence of the decision.
Cross-Layer Attention Combination. The attention matrix , collected during inference, is multi-layered, with different layers extracting distinct levels of features [40, 41]. To obtain a holistic view of the model’s attention from input to output, it is necessary to aggregate the attention maps from each layer. Substantial research indicates that while lower layers capture syntactic features and higher layers manage task-specific integration [42, 43, 44], the middle layers are predominantly responsible for the complex semantic relationships and inferential reasoning that are critical for our analysis [45, 46]. Therefore, to better extract decision-making features, we apply a Gaussian-weighted aggregation to the per-layer attention matrices, prioritizing the model’s central layers to construct a unified attention map that highlights the semantic features most critical for reasoning. In Appendix 8, we provide a detailed analysis and comparison of alternative methods for cross-layer attention feature extraction.
Sink Filter. The attention sink [34, 35] refers to the tendency of certain tokens, which often have little task-specific semantic meaning, to attract a high attention activation. This phenomenon manifests as an unavoidable stochastic noise, thereby hindering the use of attention as a reliable signal [47]. The goal of our Sink Filter (Algorithm 2) is to distinguish and remove these architectural artifacts, isolating the activations that represent the true decision-relevant dependency evidence. Specifically, we design a two-stage attention sink filter algorithm (Algorithm 2), leveraging two core features of the sink phenomenon: an abnormally high cumulative activation and a highly uniform distribution of received attention. This algorithm takes the cross-layer combined attention matrix as input. We perform the first-stage screening by identifying the top-k input tokens with the highest cumulative activation (total received attention). Then, to distinguish true architectural sinks from tokens that are merely semantically important, we compute the normalized entropy for each of these candidates, where a high entropy value indicates a uniform, sink-like distribution. This two-stage filtering mechanism maintains strong hyperparameter robustness, as demonstrated in Figure 7.
Attention Partition. Based on token mapping , the Attention Partition locates the positions of logical vertices within the attention matrix. It then partitions matrix into a set of disjoint sub-matrices, each denoted as , where the rows correspond to the tokens of a target vertex (the output), and the columns correspond to the tokens of a source vertex (the input). Finally, for each valid , we add a corresponding edge (, ) to the graph.
TAE Aggregation. The final step aggregates the attention for each edge into a single, robust metric. As established in §3.2, our goal is to quantify the total influence energy of the implicit code execution, not merely a simple signal average. We therefore use the Total Attention Energy (TAE) (defined in §3.2, Eq. (3)), which is the sum of squared attention scores. By squaring the activations to enhance SNR [48, 49],giving significantly more weight to the strong dependency signals (the code being executed) while de-emphasizing the low-value activations (the data being passively read).
5.3 *Decision Integrity Defender *
The Decision Integrity Defender module implements the Decision Integrity Verification workflow, fulfilling the defense goals outlined in §3. It analyzes the DDG constructed by the Dependency Graph Builder to detect and attribute CFI/DFI violations upon generating a tool call.
Integrity Anomaly Checking. The Integrity Anomaly Checking module performs real-time detection and attribution of integrity violations by analyzing anomalous dependencies in the Decision Dependency Graph (DDG). It operates in a policy-agnostic manner, directly inspecting the structural integrity of decision provenance as defined in §4.3. To enable a unified decision integrity verification analysis across heterogeneous contexts (e.g., varying tool quantities and context lengths), we introduce a more robust, context-adaptive metric for edges of all uninvoked tools, which we term the Decision Integrity Ratio (DIR):
[TABLE]
where is the logical vertex for the user query, and is the input context vertex corresponding to the invoked tool. This metric quantifies the relative influence of untrusted data on the final decision compared to legitimate decision sources: user query and invoked tool metadata. An edge is identified as anomalous if its DIR exceeds a single threshold : Poisoned call is detected (Detect) and is the poisoned source (Attribute). Experimental results in Appendix 8 show the effectiveness of the DIR score.
6 Evaluation
We evaluate MindGuard on various LLM agents using diverse applications under both fully adversarial attack datasets and realistic settings (normal calls contain no poisoned descriptions). Our highlights are as follows:
MindGuard successfully achieves our security goals, demonstrating 97%+ accuracy in identifying poisoned calls and 98%+ accuracy in tracing them to the poisoned source, while highly adaptable across various LLMs (§6.2).
As an intrinsic self-inspection mechanism, MindGuard requires no external resources. Its security performance outperforms SOTA methods while saving over 1000+ token overhead for each invocation (§6.3).
MindGuard demonstrates robust effectiveness with respect to involved variables (§6.4) and maintains resilience against adaptive attackers (§6.5).
6.1 Experimental Setup
Dataset and LLM Agent. To evaluate the performance of MindGuard, we curated three distinct attack datasets: 1) MCPTox [9]. This dataset consists of tools collected from real-world MCP scenarios, encompassing MCP servers with varying tool quantities across diverse settings. All samples within this dataset are metadata-poisoned, covering both CFI violation and DFI violation attack types. 2) InjecAgent [50] and 3) RAS-Eval [51]. These two datasets exclusively contain CFI violation attacks, originally designed for indirect prompt injection. We extracted the malicious payloads from these datasets, transforming them into metadata poisoning samples based on the principle established in [9]. We then evaluate the effectiveness and generalizability of MindGuard across multiple LLM agents, including models from the Qwen [52], Phi [53], Mistral [54], and Gemma [55] families. We also validate our method’s generalization by testing on models using Chain-of-Thought (CoT) reasoning (flagged with †). Crucially, MindGuard succeeds without examining the specific reasoning content, instead constructing the DDG solely from the final tool-call decision. The label distribution for each LLM agent is presented in Table V. Detailed descriptions of dataset processing and labeling procedures can be found in Appendix 8.
Hyperparameter. We set fixed hyperparameters of and for the sink filter across different LLM agents. The final anomaly detection threshold, , is designed to be tunable according to security requirements.
Baseline. To the best of our knowledge, MindGuard is the first decision guardrail against metadata poisoning. Therefore, to comprehensively evaluate its performance and justify our design choices, we conduct a comparison against the following ablated versions of MindGuard: 1) w/o Filter: The complete MindGuard system but with the attention sink filtering stage removed. 2) One-stage Filter: replacing our two-stage sink filter with one-stage filter that relies solely on cumulative activation. 3) Sum Aggregation: replacing TAE aggregation with a simple linear sum aggregator. 4) Unified Vertex: combining the partitioned and as a single vertex. For a comprehensive comparison, we also evaluate MindGuard against State-of-the-Art (SOTA) mechanisms across four distinct categories: 1) Static Content Scanning, including LLM Detector [27] and LLM Guard [16]; 2) Architectural Isolation, including CaMeL [18]; 3) Invocation Behavior Auditing, including MELON [21] and MCIP [20]; and 4) Traditional Prompt Injection Detection Methods, such as Attention Tracker [56]. Comprehensive implementation details for all baseline methods are provided in Appendix .1.
Metrics. In line with our security goals defined in §3.3, we evaluate two distinct tasks: 1) Detection task answers the question: Is the generated LLM tool call a poisoned call?; 2) Attribution task answers the question: When an attack is detected, which tool’s metadata is the source of the poisoning? Specially, for the detection task, we measure overall performance using: Accuracy; Average Precision, the area under the Precision-Recall Curve, which is the primary metric for imbalanced-class settings; and , the area under the ROC Curve. For the attribution task, we introduce Attribution Accuracy , defined as the ratio of correctly attributed attacks to the total number of detected attacks. Notably, our goal is to detect dynamic decision integrity violations (poisoned call) rather than identifying poisoned content in the context itself. Accordingly, we define Positive Samples as those Poisoned calls and Negative samples as those Normal calls, even if they may have a poisoned input context.
6.2 Overall Performance
We evaluate the overall performance of MindGuard across a diverse set of LLM agents, with results summarized in Table VI. The evaluation demonstrates that our approach is both highly effective and broadly generalizable.
Performance on Detecting Poisoned Calls. First, we evaluate our method in a challenging adversarial setting where all samples, including normal calls, reside within a poisoned context. In this setting, MindGuard shows a strong ability to identify attacks while maintaining a low false alarm rate, reflected in an average AP score of 97.6% and AUC score of 99.1%. Moreover, in the more practical scenario represented by Clean, where negative samples are drawn from a completely benign context, MindGuard’s performance is even stronger for all tested models. Notably, falls below 90% for some models (e.g., Mistral), primarily stemming from a low inherent attack success rate, thus even a few false negatives significantly impact the score.
Performance on Attributing Poisoned Metadata. MindGuard demonstrates near-perfect performance, with the attribution accuracy is 100% on multiple models, including Qwen3-8b, Qwen3-14b, and Gemma2-9b. The average attribution accuracy across all models is over 98%, validating that MindGuard can not only detect an attack but also precisely trace it back to the specific poisoned tool.
Fine-Grained CFI/DFI Performance Analysis. To evaluate MindGuard against different CFI and DFI integrity violations attacks, we decouple its detection mechanism and report performance in Table VII (evaluated on MCPTox). MindGuard achieves better TPR/FPR trade-off for both types. However, we find CFI detection performs slightly below DFI and requires a lower optimal threshold. The primary reason is that tool selection involves subtle, distributed semantic reasoning, whereas argument generation exhibits strong, localized attention patterns stemming from string copying, which makes DFI violations more readily detectable. Given that real-world attacks often combine both violation types, we jointly inspect CFI and DFI.
6.3 Comparisons with Other Works
The reference list from the paper itself. Each links out to its DOI / PubMed record.
- 1[1] J. Luo, W. Zhang, Y. Yuan, Y. Zhao, J. Yang, Y. Gu, B. Wu, B. Chen, Z. Qiao, Q. Long, R. Tu, X. Luo, W. Ju, Z. Xiao, Y. Wang, M. Xiao, C. Liu, J. Yuan, S. Zhang, Y. Jin, F. Zhang, X. Wu, H. Zhao, D. Tao, P. S. Yu, and M. Zhang, “Large language model agent: A survey on methodology, applications and challenges,” 2025. [Online]. Available: https://arxiv.org/abs/2503.21460
- 2[2] A. Yehudai, L. Eden, A. Li, G. Uziel, Y. Zhao, R. Bar-Haim, A. Cohan, and M. Shmueli-Scheuer, “Survey on evaluation of llm-based agents,” 2025. [Online]. Available: https://arxiv.org/abs/2503.16416
- 3[3] C. Qu, S. Dai, X. Wei, H. Cai, S. Wang, D. Yin, J. Xu, and J.-r. Wen, “Tool learning with large language models: a survey,” Frontiers of Computer Science , vol. 19, no. 8, Jan. 2025. [Online]. Available: http://dx.doi.org/10.1007/s 11704-024-40678-2
- 4[4] L. Matan. (2025, 09) The trifecta: How three new gemini vulnerabilities in cloud assist, search model, and browsing allowed private data exfiltration. Tenable. [Online]. Available: https://f 5.pm/go-366568.html
- 5[5] L. A. Romain Gaucher, Jayson De Lancey, “Security alert — nx compromised to steal wallets and credentials,” https://semgrep.dev/blog/2025/security-alert-nx-compromised-to-steal-wallets-and-credentials/, 2025, [Online; accessed 2025-10-17].
- 6[6] Prompt Armor, “Data exfiltration from Slack AI via indirect prompt injection,” https://promptarmor.substack.com/p/data-exfiltration-from-slack-ai-via, Aug. 2024, substack.
- 7[7] OWASP Foundation, “Code injection,” https://owasp.org/www-community/attacks/Code_Injection, accessed: 2025-11-11.
- 8[8] J. Clarke-Salt, SQL injection attacks and defense . Elsevier, 2009.
