Sparse but Strong: Crafting Adversarially Robust Graph Lottery Tickets

TL;DR

This paper introduces a framework for creating sparse graph neural networks that are highly resistant to adversarial structure perturbations, maintaining performance while reducing computational costs.

Contribution

The paper proposes ARGS, a novel adversarially robust graph sparsification method that enhances the robustness of graph lottery tickets against structure attacks.

Findings

GLTs are vulnerable to structure perturbations

ARGS improves robustness of GLTs under various attacks

Sparse GLTs maintain competitive accuracy with high robustness

Abstract

Graph Lottery Tickets (GLTs), comprising a sparse adjacency matrix and a sparse graph neural network (GNN), can significantly reduce the inference latency and compute footprint compared to their dense counterparts. Despite these benefits, their performance against adversarial structure perturbations remains to be fully explored. In this work, we first investigate the resilience of GLTs against different structure perturbation attacks and observe that they are highly vulnerable and show a large drop in classification accuracy. Based on this observation, we then present an adversarially robust graph sparsification (ARGS) framework that prunes the adjacency matrix and the GNN weights by optimizing a novel loss function capturing the graph homophily property and information associated with both the true labels of the train nodes and the pseudo labels of the test nodes. By iteratively applying ARGS to prune both the perturbed graph adjacency matrix and the GNN model weights, we can find adversarially robust graph lottery tickets that are highly sparse yet achieve competitive performance under different untargeted training-time structure attacks. Evaluations conducted on various benchmarks, considering different poisoning structure attacks, namely, PGD, MetaAttack, Meta-PGD, and PR-BCD demonstrate that the GLTs generated by ARGS can significantly improve the robustness, even when subjected to high levels of sparsity.