LMBot: Distilling Graph Knowledge into Language Model for Graph-less Deployment in Twitter Bot Detection

TL;DR

LMBot is a novel framework that distills graph neural network knowledge into language models, enabling effective graph-less Twitter bot detection with improved robustness, efficiency, and versatility.

Contribution

The paper introduces LMBot, a method that transfers GNN knowledge into language models for graph-free deployment in Twitter bot detection, addressing data dependency and bias issues.

Findings

LMBot achieves state-of-the-art results on four benchmarks.

LMBot is more robust and versatile than traditional graph-based methods.

LMBot reduces inference time by eliminating the need for graph data.

Abstract

As malicious actors employ increasingly advanced and widespread bots to disseminate misinformation and manipulate public opinion, the detection of Twitter bots has become a crucial task. Though graph-based Twitter bot detection methods achieve state-of-the-art performance, we find that their inference depends on the neighbor users multi-hop away from the targets, and fetching neighbors is time-consuming and may introduce bias. At the same time, we find that after finetuning on Twitter bot detection, pretrained language models achieve competitive performance and do not require a graph structure during deployment. Inspired by this finding, we propose a novel bot detection framework LMBot that distills the knowledge of graph neural networks (GNNs) into language models (LMs) for graph-less deployment in Twitter bot detection to combat the challenge of data dependency. Moreover, LMBot is compatible with graph-based and graph-less datasets. Specifically, we first represent each user as a textual sequence and feed them into the LM for domain adaptation. For graph-based datasets, the output of LMs provides input features for the GNN, enabling it to optimize for bot detection and distill knowledge back to the LM in an iterative, mutually enhancing process. Armed with the LM, we can perform graph-less inference, which resolves the graph data dependency and sampling bias issues. For datasets without graph structure, we simply replace the GNN with an MLP, which has also shown strong performance. Our experiments demonstrate that LMBot achieves state-of-the-art performance on four Twitter bot detection benchmarks. Extensive studies also show that LMBot is more robust, versatile, and efficient compared to graph-based Twitter bot detection methods.