Fake News Spreader Detection on Twitter using Character N-Grams. Notebook for PAN at CLEF 2020

TL;DR

This paper presents a multilingual approach using character n-grams and machine learning to identify Twitter users who spread fake news, achieving competitive accuracy and ranking third in a PAN challenge.

Contribution

It introduces a novel profiling system for fake news spreader detection on Twitter using character n-grams and machine learning models for English and Spanish.

Findings

Achieved 73% accuracy in English and 79% in Spanish.

Character n-grams are effective features for user profiling.

The task remains challenging with room for further research.

Abstract

The authors of fake news often use facts from verified news sources and mix them with misinformation to create confusion and provoke unrest among the readers. The spread of fake news can thereby have serious implications on our society. They can sway political elections, push down the stock price or crush reputations of corporations or public figures. Several websites have taken on the mission of checking rumors and allegations, but are often not fast enough to check the content of all the news being disseminated. Especially social media websites have offered an easy platform for the fast propagation of information. Towards limiting fake news from being propagated among social media users, the task of this year's PAN 2020 challenge lays the focus on the fake news spreaders. The aim of the task is to determine whether it is possible to discriminate authors that have shared fake news in the past from those that have never done it. In this notebook, we describe our profiling system for the fake news detection task on Twitter. For this, we conduct different feature extraction techniques and learning experiments from a multilingual perspective, namely English and Spanish. Our final submitted systems use character n-grams as features in combination with a linear SVM for English and Logistic Regression for the Spanish language. Our submitted models achieve an overall accuracy of 73% and 79% on the English and Spanish official test set, respectively. Our experiments show that it is difficult to differentiate solidly fake news spreaders on Twitter from users who share credible information leaving room for further investigations. Our model ranked 3rd out of 72 competitors.