Exact Certification of (Graph) Neural Networks Against Label Poisoning
Mahalakshmi Sabanayagam, Lukas Gosch, Stephan G\"unnemann and, Debarghya Ghoshdastidar
TL;DR
This paper introduces the first exact certification method for Graph Neural Networks against label poisoning, leveraging Neural Tangent Kernel and MILP to guarantee robustness and analyze architectural effects.
Contribution
The authors develop an exact certification approach for GNNs against label flipping using NTK and MILP, addressing a previously unsolved problem.
Findings
Established hierarchies of GNN robustness on benchmark graphs.
Quantified the impact of architecture choices like activations and skip-connections.
Discovered robustness plateau phenomenon at intermediate perturbation budgets.
Abstract
Machine learning models are highly vulnerable to label flipping, i.e., the adversarial modification (poisoning) of training labels to compromise performance. Thus, deriving robustness certificates is important to guarantee that test predictions remain unaffected and to understand worst-case robustness behavior. However, for Graph Neural Networks (GNNs), the problem of certifying label flipping has so far been unsolved. We change this by introducing an exact certification method, deriving both sample-wise and collective certificates. Our method leverages the Neural Tangent Kernel (NTK) to capture the training dynamics of wide networks enabling us to reformulate the bilevel optimization problem representing label flipping into a Mixed-Integer Linear Program (MILP). We apply our method to certify a broad range of GNN architectures in node classification tasks. Thereby, concerning the…
Peer Reviews
Decision·ICLR 2025 Spotlight
1. The robustness certification is a vital problem for GNN and other NNs. It is hard to obtain an exact robustness certification. 2. The author proposes a novel framework for robustness certification against label poisoning, and their main idea is to approximate the model by its NTK. It seems to be easy to apply this framework to other models. 3. The computational complexity of solving the MILP is not very large. The authors conduct experiments on several datasets and GNNs to show its empirical
1. My main concern is about the approximation error when using NTK to approximate GNN. The approximation error exists unless the model is infinitely wide, especially when we use a pooling layer. However, the authors aim to obtain exact certificates, but the approximation error is not considered in this work. 2. A limitation of this framework is that it requires the width of the model to be sufficiently large, and it cannot be used for narrow NNs. 3. An important recent work is [a], which studie
1. The authors introduce a novel exact robustness certification method for label flipping attacks on neural networks, particularly Graph Neural Networks (GNNs). This work is impactful given that exact certification for poisoning attacks is generally unsolved for GNNs. 2. By leveraging the Neural Tangent Kernel (NTK), the paper rigorously reformulates the robustness certification problem as a Mixed-Integer Linear Program (MILP). The authors were able to extend previous work [1], deriving compl
1. The high complexity of Mixed-Integer Linear Program (MILP) could introduce computational overhead, which may be prohibitive for real-time or large-scale applications. 2. Additionally, the certification method is largely tested on synthetic datasets. Although effective on these datasets, it’s unclear how well it would generalize to real-world graphs. 3. Finally, the paper focuses solely on exact certification. The authors are encouraged to consider trade-offs between exactness and scalabilit
**The Problem Addressed is Highly Relevant** Poisoning attacks, of which label flipping is one variant, are highly relevant in machine learning and pose a significant security risk. While empirical defenses can sometimes weaken this effect, it is prudent to develop methods that can guarantee their absence and accurately measure their influence. The paper thus addresses a key challenge to develop more secure and trustworthy models. **The Proposed Method is Sound Under The Infinite-Width Assumpt
**The “Exact” Certificates are Approximations** The authors claim exact certificates (e.g., L68), which means the certificate has neither false positives nor false negatives. However, this only holds under the assumption of infinite-width models, and not for finite-width models used in practice. While I am not an expert in NTKs, my understanding is that there can be significant deviations in training behavior, especially for deeper models. I would therefore argue that the framing of the certifi
Code & Models
Videos
Taxonomy
TopicsComputational Drug Discovery Methods
MethodsNeural Tangent Kernel · Focus