Beyond Detection: Exploring Evidence-based Multi-Agent Debate for Misinformation Intervention and Persuasion

TL;DR

This paper introduces ED2D, a multi-agent debate framework that uses evidence retrieval to improve misinformation detection and persuasion, outperforming baselines and highlighting both benefits and risks of MAD systems.

Contribution

ED2D extends multi-agent debate frameworks by integrating factual evidence retrieval and designing for both detection and persuasion, with comparative analysis against human experts.

Findings

ED2D outperforms existing baselines on three misinformation detection benchmarks.

When correct, ED2D's debunking transcripts are as persuasive as human experts.

Misclassification by ED2D can reinforce misconceptions despite explanations.

Abstract

Multi-agent debate (MAD) frameworks have emerged as promising approaches for misinformation detection by simulating adversarial reasoning. While prior work has focused on detection accuracy, it overlooks the importance of helping users understand the reasoning behind factual judgments and develop future resilience. The debate transcripts generated during MAD offer a rich but underutilized resource for transparent reasoning. In this study, we introduce ED2D, an evidence-based MAD framework that extends previous approach by incorporating factual evidence retrieval. More importantly, ED2D is designed not only as a detection framework but also as a persuasive multi-agent system aimed at correcting user beliefs and discouraging misinformation sharing. We compare the persuasive effects of ED2D-generated debunking transcripts with those authored by human experts. Results demonstrate that ED2D outperforms existing baselines across three misinformation detection benchmarks. When ED2D generates correct predictions, its debunking transcripts exhibit persuasive effects comparable to those of human experts; However, when ED2D misclassifies, its accompanying explanations may inadvertently reinforce users'misconceptions, even when presented alongside accurate human explanations. Our findings highlight both the promise and the potential risks of deploying MAD systems for misinformation intervention. We further develop a public community website to help users explore ED2D, fostering transparency, critical thinking, and collaborative fact-checking.