Lifelong Evolution: Collaborative Learning between Large and Small Language Models for Continuous Emergent Fake News Detection

TL;DR

This paper introduces a collaborative learning framework combining large and small language models to improve fake news detection, addressing challenges of data scarcity, evolving news, and knowledge updates with continual learning techniques.

Contribution

It proposes a novel lifelong collaborative framework with knowledge editing and replay-based learning to enhance fake news detection in dynamic environments.

Findings

Significantly outperforms existing methods on Pheme and Twitter16 datasets.

Improves detection accuracy and adaptability in continuous fake news scenarios.

Effectively updates models without complete retraining.

Abstract

The widespread dissemination of fake news on social media has significantly impacted society, resulting in serious consequences. Conventional deep learning methodologies employing small language models (SLMs) suffer from extensive supervised training requirements and difficulties adapting to evolving news environments due to data scarcity and distribution shifts. Large language models (LLMs), despite robust zero-shot capabilities, fall short in accurately detecting fake news owing to outdated knowledge and the absence of suitable demonstrations. In this paper, we propose a novel Continuous Collaborative Emergent Fake News Detection (C$^2$EFND) framework to address these challenges. The C$^2$EFND framework strategically leverages both LLMs' generalization power and SLMs' classification expertise via a multi-round collaborative learning framework. We further introduce a lifelong knowledge editing module based on a Mixture-of-Experts architecture to incrementally update LLMs and a replay-based continue learning method to ensure SLMs retain prior knowledge without retraining entirely. Extensive experiments on Pheme and Twitter16 datasets demonstrate that C$^2$EFND significantly outperforms existed methods, effectively improving detection accuracy and adaptability in continuous emergent fake news scenarios.