Goal-Aware Identification and Rectification of Misinformation in Multi-Agent Systems
Zherui Li, Yan Mi, Zhenhong Zhou, Houcheng Jiang, Guibin Zhang, Kun Wang, Junfeng Fang
TL;DR
This paper introduces MisinfoTask, a dataset for evaluating misinformation in multi-agent systems, and ARGUS, a goal-aware, training-free framework that effectively reduces misinformation and enhances task success under attack.
Contribution
The paper presents a novel dataset MisinfoTask and a new defense framework ARGUS that improves misinformation rectification without additional training.
Findings
ARGUS reduces misinformation toxicity by ~28.17%.
ARGUS improves task success rates under attack by ~10.33%.
Demonstrates effectiveness across various injection attacks.
Abstract
Large Language Model-based Multi-Agent Systems (MASs) have demonstrated strong advantages in addressing complex real-world tasks. However, due to the introduction of additional attack surfaces, MASs are particularly vulnerable to misinformation injection. To facilitate a deeper understanding of misinformation propagation dynamics within these systems, we introduce MisinfoTask, a novel dataset featuring complex, realistic tasks designed to evaluate MAS robustness against such threats. Building upon this, we propose ARGUS, a two-stage, training-free defense framework leveraging goal-aware reasoning for precise misinformation rectification within information flows. Our experiments demonstrate that in challenging misinformation scenarios, ARGUS exhibits significant efficacy across various injection attacks, achieving an average reduction in misinformation toxicity of approximately 28.17%…
Peer Reviews
Decision·ICLR 2026 Poster
1. The paper is studying a meaningful question that how misinformation propagate within MAS after information injection attacks. 2. The paper builds a complete benchmark including the dataset, setup and evaluation, and also propose a method to tackle such problem.
1. The paper first uses LLM to generate tasks, and then manually filter out tasks. However, it is unclear whether these tasks align with real-world tasks and whether they are diverse enough to be used, since the same prompt is being used to generate tasks over and over. 2. The dataset only contains 108 tasks, which is small, especially the main content is crafted by LLM. 3. The evaluation employs an LLM judge. Although LLM judge could be useful, the paper doesn't have any sanity check of it, for
- The design is original. The synthesis of static topological analysis with dynamic, semantic re-localization based on an inferred misinformation goal is a clever and novel approach. Furthermore, the concept of a "persuasive" corrective agent using CoT reasoning is more advanced than simple fact-checking or edge-pruning defenses. - The MISINFOTASK dataset is a useful contribution. - Experiments are thorough. This paper tests across multiple LLM families, multiple modern attack vectors and d
- The paper acknowledges the limit of computational overhead & cost but does not quantify the overhead. - The initial localization step relies on Edge Betweenness Centrality, which is computationally expensive ($O(V \cdot E)$) and does not scale well to large graphs. This would be a bottleneck for MAS with tens or hundreds of agents. - The paper's definition of misinformation is "content that contradicts the factual knowledge implicitly stored in the parameters of an LLM." Thus "Internal Knowl
1. The paper introduces a novel and practical dataset that enables rigorous evaluation of misinformation robustness in multi-agent systems. 2. The proposed ARGUS framework effectively mitigates misinformation without additional training and improves task completion. 3. The work addresses an important and timely problem of misinformation security that has been largely overlooked in prior MAS research. 4. The paper is clearly written, well organized, and supported by comprehensive experimental val
1. It is recommended that the authors revise the structure of Table 1 so that the model names appear in the first column and the defense types in the second column, making the table layout clearer and improving comparability across models. 2. The paper would benefit from a clearer specification of the threat model, detailing attacker goals, capabilities, and assumptions, which would help strengthen the discussion on the security significance of misinformation propagation in MAS. 3. In MAS, var
Code & Models
Videos
Taxonomy
TopicsMulti-Criteria Decision Making · Bayesian Modeling and Causal Inference · Information and Cyber Security
MethodsMixing Adam and SGD