Exploring Health Misinformation Detection with Multi-Agent Debate

TL;DR

This paper introduces a two-stage framework combining large language model-based agreement scoring and multi-agent debate to improve health misinformation detection, demonstrating superior performance over baseline methods.

Contribution

The paper presents a novel two-stage approach integrating agreement scoring and multi-agent debate for more effective health misinformation verification.

Findings

The approach outperforms baseline methods in accuracy.

Multi-agent debate enhances evidence synthesis.

Agreement score prediction guides the debate process.

Abstract

Fact-checking health-related claims has become increasingly critical as misinformation proliferates online. Effective verification requires both the retrieval of high-quality evidence and rigorous reasoning processes. In this paper, we propose a two-stage framework for health misinformation detection: Agreement Score Prediction followed by Multi-Agent Debate. In the first stage, we employ large language models (LLMs) to independently evaluate retrieved articles and compute an aggregated agreement score that reflects the overall evidence stance. When this score indicates insufficient consensus-falling below a predefined threshold-the system proceeds to a second stage. Multiple agents engage in structured debate to synthesize conflicting evidence and generate well-reasoned verdicts with explicit justifications. Experimental results demonstrate that our two-stage approach achieves superior performance compared to baseline methods, highlighting the value of combining automated scoring with collaborative reasoning for complex verification tasks.