Simplistic Collection and Labeling Practices Limit the Utility of Benchmark Datasets for Twitter Bot Detection

TL;DR

This paper demonstrates that the high performance of existing Twitter bot detection datasets is largely due to simplistic collection and labeling practices, which limit their generalizability and reliability for research and practical use.

Contribution

It reveals that current datasets' collection and labeling methods inflate performance metrics, highlighting the need for more robust and transparent practices in bot detection research.

Findings

Simple decision rules perform nearly as well as complex models on existing datasets.

Datasets do not generalize well to out-of-sample data, indicating overfitting.

Performance heavily depends on dataset collection and labeling procedures.

Abstract

Accurate bot detection is necessary for the safety and integrity of online platforms. It is also crucial for research on the influence of bots in elections, the spread of misinformation, and financial market manipulation. Platforms deploy infrastructure to flag or remove automated accounts, but their tools and data are not publicly available. Thus, the public must rely on third-party bot detection. These tools employ machine learning and often achieve near perfect performance for classification on existing datasets, suggesting bot detection is accurate, reliable and fit for use in downstream applications. We provide evidence that this is not the case and show that high performance is attributable to limitations in dataset collection and labeling rather than sophistication of the tools. Specifically, we show that simple decision rules -- shallow decision trees trained on a small number of features -- achieve near-state-of-the-art performance on most available datasets and that bot detection datasets, even when combined together, do not generalize well to out-of-sample datasets. Our findings reveal that predictions are highly dependent on each dataset's collection and labeling procedures rather than fundamental differences between bots and humans. These results have important implications for both transparency in sampling and labeling procedures and potential biases in research using existing bot detection tools for pre-processing.