Multi-Agent Retrieval-Augmented Framework for Evidence-Based Counterspeech Against Health Misinformation

TL;DR

This paper introduces a multi-agent retrieval-augmented framework that leverages multiple large language models to generate relevant, accurate, and well-grounded counterspeech against health misinformation, improving quality and generalization.

Contribution

It presents a novel multi-agent framework integrating static and dynamic evidence to enhance counterspeech generation, outperforming baseline methods in multiple quality metrics.

Findings

Outperforms baseline approaches in politeness, relevance, informativeness, and factual accuracy.

Ablation studies confirm the importance of each framework component.

System generalizes well across diverse health misinformation topics.

Abstract

Large language models (LLMs) incorporated with Retrieval-Augmented Generation (RAG) have demonstrated powerful capabilities in generating counterspeech against misinformation. However, current studies rely on limited evidence and offer less control over final outputs. To address these challenges, we propose a Multi-agent Retrieval-Augmented Framework to generate counterspeech against health misinformation, incorporating multiple LLMs to optimize knowledge retrieval, evidence enhancement, and response refinement. Our approach integrates both static and dynamic evidence, ensuring that the generated counterspeech is relevant, well-grounded, and up-to-date. Our method outperforms baseline approaches in politeness, relevance, informativeness, and factual accuracy, demonstrating its effectiveness in generating high-quality counterspeech. To further validate our approach, we conduct ablation studies to verify the necessity of each component in our framework. Furthermore, cross evaluations show that our system generalizes well across diverse health misinformation topics and datasets. And human evaluations reveal that refinement significantly enhances counterspeech quality and obtains human preference.