Breaking Event Rumor Detection via Stance-Separated Multi-Agent Debate

TL;DR

This paper introduces S2MAD, a multi-agent debate framework that leverages stance separation and structured debate to improve rumor detection during breaking events, enhancing LLM performance in complex, real-world scenarios.

Contribution

The paper presents a novel multi-agent debate approach with stance separation and structured argumentation to improve rumor detection accuracy during breaking events.

Findings

Outperforms state-of-the-art methods on real-world datasets.

Effectively improves LLM performance in rumor detection.

Enhances detection accuracy for complex and controversial claims.

Abstract

The rapid spread of rumors on social media platforms during breaking events severely hinders the dissemination of the truth. Previous studies reveal that the lack of annotated resources hinders the direct detection of unforeseen breaking events not covered in yesterday's news. Leveraging large language models (LLMs) for rumor detection holds significant promise. However, it is challenging for LLMs to provide comprehensive responses to complex or controversial issues due to limited diversity. In this work, we propose the Stance Separated Multi-Agent Debate (S2MAD) to address this issue. Specifically, we firstly introduce Stance Separation, categorizing comments as either supporting or opposing the original claim. Subsequently, claims are classified as subjective or objective, enabling agents to generate reasonable initial viewpoints with different prompt strategies for each type of claim. Debaters then follow specific instructions through multiple rounds of debate to reach a consensus. If a consensus is not reached, a judge agent evaluates the opinions and delivers a final verdict on the claim's veracity. Extensive experiments conducted on two real-world datasets demonstrate that our proposed model outperforms state-of-the-art methods in terms of performance and effectively improves the performance of LLMs in breaking event rumor detection.