TrollHunter [Evader]: Automated Detection [Evasion] of Twitter Trolls During the COVID-19 Pandemic

TL;DR

This paper introduces TrollHunter, an automated system for detecting Twitter trolls during COVID-19 with high accuracy, and explores how trolls can evade detection using adversarial machine learning techniques.

Contribution

The paper presents a novel linguistic analysis-based detection method and an adversarial evasion mechanism, advancing the understanding of troll detection and evasion strategies during pandemics.

Findings

TrollHunter achieved 98.5% accuracy in detecting trolls.

TrollHunter-Evader reduced detection accuracy by 40%.

The study discusses implications of adversarial evasion in misinformation control.

Abstract

This paper presents TrollHunter, an automated reasoning mechanism we used to hunt for trolls on Twitter during the COVID-19 pandemic in 2020. Trolls, poised to disrupt the online discourse and spread disinformation, quickly seized the absence of a credible response to COVID-19 and created a COVID-19 infodemic by promulgating dubious content on Twitter. To counter the COVID-19 infodemic, the TrollHunter leverages a unique linguistic analysis of a multi-dimensional set of Twitter content features to detect whether or not a tweet was meant to troll. TrollHunter achieved 98.5% accuracy, 75.4% precision and 69.8% recall over a dataset of 1.3 million tweets. Without a final resolution of the pandemic in sight, it is unlikely that the trolls will go away, although they might be forced to evade automated hunting. To explore the plausibility of this strategy, we developed and tested an adversarial machine learning mechanism called TrollHunter-Evader. TrollHunter-Evader employs a Test Time Evasion (TTE) approach in a combination with a Markov chain-based mechanism to recycle originally trolling tweets. The recycled tweets were able to achieve a remarkable 40% decrease in the TrollHunter's ability to correctly identify trolling tweets. Because the COVID-19 infodemic could have a harmful impact on the COVID-19 pandemic, we provide an elaborate discussion about the implications of employing adversarial machine learning to evade Twitter troll hunts.