PCoT: Persuasion-Augmented Chain of Thought for Detecting Fake News and Social Media Disinformation

TL;DR

This paper introduces PCoT, a novel method that uses persuasion knowledge to enhance zero-shot disinformation detection in social media, demonstrating significant improvements across multiple datasets and language models.

Contribution

The paper proposes PCoT, a new approach that incorporates persuasion awareness into chain of thought prompting, and provides two new datasets for evaluating disinformation detection.

Findings

PCoT outperforms existing methods by 15% on average.

The approach improves detection across five different LLMs.

New datasets enable evaluation on unseen, post-publication content.

Abstract

Disinformation detection is a key aspect of media literacy. Psychological studies have shown that knowledge of persuasive fallacies helps individuals detect disinformation. Inspired by these findings, we experimented with large language models (LLMs) to test whether infusing persuasion knowledge enhances disinformation detection. As a result, we introduce the Persuasion-Augmented Chain of Thought (PCoT), a novel approach that leverages persuasion to improve disinformation detection in zero-shot classification. We extensively evaluate PCoT on online news and social media posts. Moreover, we publish two novel, up-to-date disinformation datasets: EUDisinfo and MultiDis. These datasets enable the evaluation of PCoT on content entirely unseen by the LLMs used in our experiments, as the content was published after the models' knowledge cutoffs. We show that, on average, PCoT outperforms competitive methods by 15% across five LLMs and five datasets. These findings highlight the value of persuasion in strengthening zero-shot disinformation detection.