Fine-tuning is Not Fine: Mitigating Backdoor Attacks in GNNs with Limited Clean Data

TL;DR

This paper introduces GRAPHNAD, a framework that effectively mitigates backdoor attacks in GNNs using limited clean data by aligning intermediate-layer attention and relation representations, significantly reducing attack success rates.

Contribution

Proposes GRAPHNAD, a novel backdoor mitigation method for GNNs that leverages attention transfer and relation map consistency with limited clean data.

Findings

Reduces attack success rate to below 5% with only 3% clean data.

Outperforms existing fine-tuning and distillation defenses under limited data.

Effectively aligns intermediate-layer representations to neutralize backdoors.

Abstract

Graph Neural Networks (GNNs) have achieved remarkable performance through their message-passing mechanism. However, recent studies have highlighted the vulnerability of GNNs to backdoor attacks, which can lead the model to misclassify graphs with attached triggers as the target class. The effectiveness of recent promising defense techniques, such as fine-tuning or distillation, is heavily contingent on having comprehensive knowledge of the sufficient training dataset. Empirical studies have shown that fine-tuning methods require a clean dataset of 20% to reduce attack accuracy to below 25%, while distillation methods require a clean dataset of 15%. However, obtaining such a large amount of clean data is commonly impractical. In this paper, we propose a practical backdoor mitigation framework, denoted as GRAPHNAD, which can capture high-quality intermediate-layer representations in GNNs to enhance the distillation process with limited clean data. To achieve this, we address the following key questions: How to identify the appropriate attention representations in graphs for distillation? How to enhance distillation with limited data? By adopting the graph attention transfer method, GRAPHNAD can effectively align the intermediate-layer attention representations of the backdoored model with that of the teacher model, forcing the backdoor neurons to transform into benign ones. Besides, we extract the relation maps from intermediate-layer transformation and enforce the relation maps of the backdoored model to be consistent with that of the teacher model, thereby ensuring model accuracy while further reducing the influence of backdoors. Extensive experimental results show that by fine-tuning a teacher model with only 3% of the clean data, GRAPHNAD can reduce the attack success rate to below 5%.