BotArtist: Generic approach for bot detection in Twitter via semi-automatic machine learning pipeline

TL;DR

This paper presents BotArtist, a new semi-automatic machine learning pipeline for Twitter bot detection that outperforms existing methods by nearly 10% in F1-score and provides a large labeled dataset for future research.

Contribution

Introduction of BotArtist, a novel bot detection model based on user profile features, and the creation of one of the largest labeled Twitter bot datasets for research.

Findings

BotArtist outperforms state-of-the-art methods by nearly 10% in F1-score.

The dataset includes over 10 million Twitter profiles with bot/human labels.

Evaluation conducted across nine public datasets under standardized conditions.

Abstract

Twitter, as one of the most popular social networks, provides a platform for communication and online discourse. Unfortunately, it has also become a target for bots and fake accounts, resulting in the spread of false information and manipulation. This paper introduces a semi-automatic machine learning pipeline (SAMLP) designed to address the challenges associated with machine learning model development. Through this pipeline, we develop a comprehensive bot detection model named BotArtist, based on user profile features. SAMLP leverages nine distinct publicly available datasets to train the BotArtist model. To assess BotArtist's performance against current state-of-the-art solutions, we evaluate 35 existing Twitter bot detection methods, each utilizing a diverse range of features. Our comparative evaluation of BotArtist and these existing methods, conducted across nine public datasets under standardized conditions, reveals that the proposed model outperforms existing solutions by almost 10% in terms of F1-score, achieving an average score of 83.19% and 68.5% over specific and general approaches, respectively. As a result of this research, we provide one of the largest labeled Twitter bot datasets. The dataset contains extracted features combined with BotArtist predictions for 10,929,533 Twitter user profiles, collected via Twitter API during the 2022 Russo-Ukrainian War over a 16-month period. This dataset was created based on [Shevtsov et al., 2022a] where the original authors share anonymized tweets discussing the Russo-Ukrainian war, totaling 127,275,386 tweets. The combination of the existing textual dataset and the provided labeled bot and human profiles will enable future development of more advanced bot detection large language models in the post-Twitter API era.