CrediRAG: Network-Augmented Credibility-Based Retrieval for Misinformation Detection in Reddit

TL;DR

CrediRAG is a novel fake news detection model that combines language models with external knowledge bases and social network analysis to improve accuracy in identifying misinformation on Reddit.

Contribution

It introduces a network-augmented retrieval approach that integrates source credibility, social interactions, and stance analysis for enhanced fake news detection.

Findings

Achieves 11% higher F1-score over state-of-the-art methods.

Demonstrates superior performance on Reddit data with over 200,000 posts.

Provides a scalable solution for misinformation detection in social media.

Abstract

Fake news threatens democracy and exacerbates the polarization and divisions in society; therefore, accurately detecting online misinformation is the foundation of addressing this issue. We present CrediRAG, the first fake news detection model that combines language models with access to a rich external political knowledge base with a dense social network to detect fake news across social media at scale. CrediRAG uses a news retriever to initially assign a misinformation score to each post based on the source credibility of similar news articles to the post title content. CrediRAG then improves the initial retrieval estimations through a novel weighted post-to-post network connected based on shared commenters and weighted by the average stance of all shared commenters across every pair of posts. We achieve 11% increase in the F1-score in detecting misinformative posts over state-of-the-art methods. Extensive experiments conducted on curated real-world Reddit data of over 200,000 posts demonstrate the superior performance of CrediRAG on existing baselines. Thus, our approach offers a more accurate and scalable solution to combat the spread of fake news across social media platforms.