Adversarial Robustness of Graph Transformers

TL;DR

This paper investigates the adversarial robustness of Graph Transformers, introduces adaptive attack methods, and demonstrates their effectiveness in exposing vulnerabilities and improving robustness through adversarial training.

Contribution

It is the first to design adaptive, gradient-based attacks specifically for Graph Transformers and to evaluate their robustness across multiple architectures and tasks.

Findings

Graph Transformers are highly vulnerable to adversarial attacks.

Adaptive attacks significantly improve robustness when used for adversarial training.

The study provides general principles for attacking Graph Transformers with structure perturbations.

Abstract

Existing studies have shown that Message-Passing Graph Neural Networks (MPNNs) are highly susceptible to adversarial attacks. In contrast, despite the increasing importance of Graph Transformers (GTs), their robustness properties are unexplored. We close this gap and design the first adaptive attacks for GTs. In particular, we provide general design principles for strong gradient-based attacks on GTs w.r.t. structure perturbations and instantiate our attack framework for five representative and popular GT architectures. Specifically, we study GTs with specialized attention mechanisms and Positional Encodings (PEs) based on pairwise shortest paths, random walks, and the Laplacian spectrum. We evaluate our attacks on multiple tasks and perturbation models, including structure perturbations for node and graph classification, and node injection for graph classification. Our results reveal that GTs can be catastrophically fragile in many cases. Addressing this vulnerability, we show how our adaptive attacks can be effectively used for adversarial training, substantially improving robustness.