Navigating the Black Box: Leveraging LLMs for Effective Text-Level Graph Injection Attacks

TL;DR

This paper presents ATAG-LLM, a black-box graph injection attack framework for text-attributed graphs that uses large language models to generate interpretable attack texts, effectively disrupting graph structures with minimal training costs.

Contribution

The paper introduces a novel black-box GIA framework leveraging LLMs for text-level node attribute generation, improving attack interpretability and efficiency in real-world TAG scenarios.

Findings

ATAG-LLM outperforms existing attack methods in disrupting graph homophily.

The approach reduces training costs compared to surrogate model-based attacks.

Experiments validate the effectiveness of LLM-guided text generation in attack success.

Abstract

Text-attributed graphs (TAGs) integrate textual data with graph structures, providing valuable insights in applications such as social network analysis and recommendation systems. Graph Neural Networks (GNNs) effectively capture both topological structure and textual information in TAGs but are vulnerable to adversarial attacks. Existing graph injection attack (GIA) methods assume that attackers can directly manipulate the embedding layer, producing non-explainable node embeddings. Furthermore, the effectiveness of these attacks often relies on surrogate models with high training costs. Thus, this paper introduces ATAG-LLM, a novel black-box GIA framework tailored for TAGs. Our approach leverages large language models (LLMs) to generate interpretable text-level node attributes directly, ensuring attacks remain feasible in real-world scenarios. We design strategies for LLM prompting that balance exploration and reliability to guide text generation, and propose a similarity assessment method to evaluate attack text effectiveness in disrupting graph homophily. This method efficiently perturbs the target node with minimal training costs in a strict black-box setting, ensuring a text-level graph injection attack for TAGs. Experiments on real-world TAG datasets validate the superior performance of ATAG-LLM compared to state-of-the-art embedding-level and text-level attack methods.