Beyond Individual Mimicry: Constructing Human-Like Social network with Graph-Augmented LLM Agents

TL;DR

This paper introduces GraphMind, a framework that enables LLM-based social bots to mimic human-like social network structures, revealing weaknesses in current detection methods.

Contribution

It proposes a novel method for LLM-driven bots to learn and replicate human social networks, and constructs a botnet to evaluate detection algorithms.

Findings

Detection models perform worse on GraphMind-Botnet datasets.

Explicit social link construction is crucial for realistic social network generation.

Current detection mechanisms have fundamental weaknesses exposed by GraphMind-Botnet.

Abstract

Driven by large language models (LLMs), social bot can autonomously engage in local interactions, whose human-like behaviors enable them to evade social bot detection. However, while these botnets exhibit realistic local social interactions, they fail to preserve human-like social network. This is because LLM-based bots are graph-unaware and cannot coordinate over global interactions, which makes those botnets vulnerable to graph neural network (GNN)-based detection. To address this limitation, we propose GraphMind, which equips LLM-driven social bots to explicitly learn and fit human-like social network structures. Building on this foundation, we further construct GraphMind-Botnet, a LLM-driven botnet designed to evaluate the performance of existing social bot detection algorithms. Experiments on datasets derived from GraphMind-Botnet show that both text-based and graph-based detection models show substantially degraded performance in distinguishing. Our results highlight the critical role of social link construction in LLM-driven social network generation, while exposing fundamental weaknesses in existing bot detection mechanisms.