Robustness in Text-Attributed Graph Learning: Insights, Trade-offs, and New Defenses

TL;DR

This paper systematically evaluates the robustness of various text-attributed graph learning models against diverse perturbations, revealing inherent trade-offs and vulnerabilities, and proposes SFT-auto, a new framework for balanced robustness.

Contribution

It introduces a comprehensive evaluation framework for robustness in TAG learning and proposes SFT-auto, a novel method for balanced defense against textual and structural attacks.

Findings

Models exhibit robustness trade-offs between text and structure.

GNNs and RGNNs' performance depends on text encoder and attack type.

GraphLLMs are highly vulnerable to training data corruption.

Abstract

While Graph Neural Networks (GNNs) and Large Language Models (LLMs) are powerful approaches for learning on Text-Attributed Graphs (TAGs), a comprehensive understanding of their robustness remains elusive. Current evaluations are fragmented, failing to systematically investigate the distinct effects of textual and structural perturbations across diverse models and attack scenarios. To address these limitations, we introduce a unified and comprehensive framework to evaluate robustness in TAG learning. Our framework evaluates classical GNNs, robust GNNs (RGNNs), and GraphLLMs across ten datasets from four domains, under diverse text-based, structure-based, and hybrid perturbations in both poisoning and evasion scenarios. Our extensive analysis reveals multiple findings, among which three are particularly noteworthy: 1) models have inherent robustness trade-offs between text and structure, 2) the performance of GNNs and RGNNs depends heavily on the text encoder and attack type, and 3) GraphLLMs are particularly vulnerable to training data corruption. To overcome the identified trade-offs, we introduce SFT-auto, a novel framework that delivers superior and balanced robustness against both textual and structural attacks within a single model. Our work establishes a foundation for future research on TAG security and offers practical solutions for robust TAG learning in adversarial environments. Our code is available at: https://github.com/Leirunlin/TGRB.