If You Want to Be Robust, Be Wary of Initialization

TL;DR

This paper explores how weight initialization and training epochs influence the robustness of Graph Neural Networks against adversarial attacks, providing a theoretical framework and empirical validation to improve model resilience.

Contribution

It introduces a theoretical framework linking initialization strategies to adversarial robustness and demonstrates the importance of proper initialization for enhanced GNN resilience.

Findings

Proper initialization improves robustness by up to 50% against adversarial attacks.

Theoretical analysis connects weight choices and training duration to vulnerability.

Empirical results validate the framework across diverse models and datasets.

Abstract

Graph Neural Networks (GNNs) have demonstrated remarkable performance across a spectrum of graph-related tasks, however concerns persist regarding their vulnerability to adversarial perturbations. While prevailing defense strategies focus primarily on pre-processing techniques and adaptive message-passing schemes, this study delves into an under-explored dimension: the impact of weight initialization and associated hyper-parameters, such as training epochs, on a model's robustness. We introduce a theoretical framework bridging the connection between initialization strategies and a network's resilience to adversarial perturbations. Our analysis reveals a direct relationship between initial weights, number of training epochs and the model's vulnerability, offering new insights into adversarial robustness beyond conventional defense mechanisms. While our primary focus is on GNNs, we extend our theoretical framework, providing a general upper-bound applicable to Deep Neural Networks. Extensive experiments, spanning diverse models and real-world datasets subjected to various adversarial attacks, validate our findings. We illustrate that selecting appropriate initialization not only ensures performance on clean datasets but also enhances model robustness against adversarial perturbations, with observed gaps of up to 50\% compared to alternative initialization approaches.