Edge-free but Structure-aware: Prototype-Guided Knowledge Distillation from GNNs to MLPs

TL;DR

This paper introduces Prototype-Guided Knowledge Distillation (PGKD), a method to transfer structural information from GNNs to MLPs without using graph edges, improving low-latency graph task performance.

Contribution

The paper proposes a novel edge-free knowledge distillation approach that captures graph structure information from GNNs to enhance MLPs, without relying on graph edges.

Findings

PGKD outperforms baseline methods on benchmark datasets.

PGKD demonstrates robustness across various graph tasks.

The method effectively captures structural information without edge data.

Abstract

Distilling high-accuracy Graph Neural Networks (GNNs) to low-latency multilayer perceptions (MLPs) on graph tasks has become a hot research topic. However, conventional MLP learning relies almost exclusively on graph nodes and fails to effectively capture the graph structural information. Previous methods address this issue by processing graph edges into extra inputs for MLPs, but such graph structures may be unavailable for various scenarios. To this end, we propose Prototype-Guided Knowledge Distillation (PGKD), which does not require graph edges (edge-free setting) yet learns structure-aware MLPs. Our insight is to distill graph structural information from GNNs. Specifically, we first employ the class prototypes to analyze the impact of graph structures on GNN teachers, and then design two losses to distill such information from GNNs to MLPs. Experimental results on popular graph benchmarks demonstrate the effectiveness and robustness of the proposed PGKD.