Multi-Agent Debate with Memory Masking
Hongduan Tian, Xiao Feng, Ziyuan Zhao, Xiangyu Zhu, Rolan Yan, Bo Han
TL;DR
This paper introduces MAD-M^2, a multi-agent debate framework with memory masking that enhances reasoning robustness by filtering erroneous memories, leading to improved performance on reasoning benchmarks.
Contribution
It proposes a novel memory masking technique for multi-agent debate models to improve reasoning accuracy by reducing the impact of erroneous memories.
Findings
MAD-M^2 outperforms MAD in reasoning benchmarks.
The framework effectively identifies and masks erroneous memories.
Memory masking improves reasoning robustness and accuracy.
Abstract
Large language models (LLMs) have recently demonstrated impressive capabilities in reasoning tasks. Currently, mainstream LLM reasoning frameworks predominantly focus on scaling up inference-time sampling to enhance performance. In particular, among all LLM reasoning frameworks, *multi-agent debate* (MAD), which employs multiple LLMs as agents to perform reasoning in the way of multi-round debate, has emerged as a powerful reasoning paradigm since it allows agents to access previous memories to alleviate fallacious content and refine their reasoning iteratively in each debate round. However, although MAD significantly improves the reasoning capabilities of LLMs, in this paper, we observe that there remain erroneous memories, and LLM agents are vulnerable to these erroneous memories. To explore this phenomenon, we provide a theoretical insight that the performance of MAD is highly…
Peer Reviews
Decision·ICLR 2026 Poster
The paper identifies a real issue in MAD: agents can be misled by wrong memories from previous rounds, as illustrated in Figure 1 where Agent 1 initially answers correctly but is misled in Round 2 after seeing Agent 2's incorrect response. This is intuitive and well-demonstrated. MAD-M² is straightforward to implement: agents evaluate memories, generate binary masks, and reason with filtered memories. The method doesn't require additional models or complex infrastructure, making it practical fo
MAD-M² shows improvement on MATH (0.90→0.92, +2%) and AIME24 (0.37→0.50), but degrades or maintains performance on MMLU-Pro (0.83→0.83) and AIME25 (0.40→0.40). Meanwhile, MAD-M² shows improvement on MATH (0.90→0.92, +2%) and AIME24 (0.37→0.50), but degrades or maintains performance on MMLU-Pro (0.83→0.83) and AIME25 (0.40→0.40) Token consumption analysis in Table 1 shows MAD-M² consumes 50-100% more tokens than standard MAD. The cost-benefit ratio is poor, especially when standard MAD already
1. Clear motivation grounded in a theoretical gap: The authors identify an underexplored limitation in multi-agent debate frameworks — vulnerability to low-quality reasoning memories — and back it up with intuitive examples (Figure 1) and formal analysis. 2. The analysis in Section 2 provides explicit probabilistic bounds linking performance degradation to the number of wrong memories. The inclusion of both hard and easy reasoning settings (HPR/EPR) offers a nuanced interpretation of how memory
1. My biggest concerns are about approach performance. I only see the marginal performance gains in stronger models. While MAD-M2 improves certain benchmarks on easy tasks, the improvements for DeepSeek-V3 are minor or inconsistent. This suggests limited scalability to highly capable LLMs. Could the authors provide more results on diverse and powerful LLM benchmarks? 2. The proposed masking step roughly doubles token consumption in some cases, but the paper provides limited quantitative analysi
1. This paper is the first to demonstrate that Multi-Agent Debate (MAD) frameworks are vulnerable to false memories, providing theoretical proof of this phenomenon. 2. This study offers a thorough analysis of the key characteristics of the proposed MAD-M² framework in relation to existing methods.
1. The primary contribution of this paper lies in its theoretical insights rather than methodological innovations. 2. The proposed MAD-M² framework achieves state-of-the-art performance in only a limited subset of experimental scenarios, while underperforming traditional MAD or CoT-SC methods in the majority of cases. Moreover, MAD-M² incurs significantly higher token consumption compared to baseline methods. From a performance-efficiency trade-off perspective, MAD-M² does not demonstrate clear
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTopic Modeling · Explainable Artificial Intelligence (XAI) · Multimodal Machine Learning Applications