Adversarial Training for Graph Neural Networks via Graph Subspace Energy Optimization

TL;DR

This paper introduces a novel adversarial training method for GNNs based on graph subspace energy, improving robustness against topology perturbations and enhancing both adversarial and clean accuracy.

Contribution

It proposes the concept of graph subspace energy (GSE) as a robustness indicator and develops AT-GSE, an adversarial training approach utilizing GSE maximization with low-rank graph approximations.

Findings

AT-GSE outperforms state-of-the-art methods in adversarial accuracy.

AT-GSE achieves higher clean accuracy on non-perturbed graphs.

The method is effective across various datasets with different graph properties.

Abstract

Despite impressive capability in learning over graph-structured data, graph neural networks (GNN) suffer from adversarial topology perturbation in both training and inference phases. While adversarial training has demonstrated remarkable effectiveness in image classification tasks, its suitability for GNN models has been doubted until a recent advance that shifts the focus from transductive to inductive learning. Still, GNN robustness in the inductive setting is under-explored, and it calls for deeper understanding of GNN adversarial training. To this end, we propose a new concept of graph subspace energy (GSE) -- a generalization of graph energy that measures graph stability -- of the adjacency matrix, as an indicator of GNN robustness against topology perturbations. To further demonstrate the effectiveness of such concept, we propose an adversarial training method with the perturbed graphs generated by maximizing the GSE regularization term, referred to as AT-GSE. To deal with the local and global topology perturbations raised respectively by LRBCD and PRBCD, we employ randomized SVD (RndSVD) and Nystrom low-rank approximation to favor the different aspects of the GSE terms. An extensive set of experiments shows that AT-GSE outperforms consistently the state-of-the-art GNN adversarial training methods over different homophily and heterophily datasets in terms of adversarial accuracy, whilst more surprisingly achieving a superior clean accuracy on non-perturbed graphs.