Data-Free Adversarial Knowledge Distillation for Graph Neural Networks

TL;DR

This paper introduces a novel data-free adversarial knowledge distillation framework for graph neural networks, enabling model compression without requiring real data, by using a generative adversarial network to synthesize training graphs.

Contribution

It presents the first end-to-end data-free distillation method for GNNs utilizing adversarial training with a generator, teacher, and student models.

Findings

Outperforms existing data-free methods on multiple benchmarks

Effective in various graph classification tasks

Reduces dependency on real data for GNN training

Abstract

Graph neural networks (GNNs) have been widely used in modeling graph structured data, owing to its impressive performance in a wide range of practical applications. Recently, knowledge distillation (KD) for GNNs has enabled remarkable progress in graph model compression and knowledge transfer. However, most of the existing KD methods require a large volume of real data, which are not readily available in practice, and may preclude their applicability in scenarios where the teacher model is trained on rare or hard to acquire datasets. To address this problem, we propose the first end-to-end framework for data-free adversarial knowledge distillation on graph structured data (DFAD-GNN). To be specific, our DFAD-GNN employs a generative adversarial network, which mainly consists of three components: a pre-trained teacher model and a student model are regarded as two discriminators, and a generator is utilized for deriving training graphs to distill knowledge from the teacher model into the student model. Extensive experiments on various benchmark models and six representative datasets demonstrate that our DFAD-GNN significantly surpasses state-of-the-art data-free baselines in the graph classification task.