From Model to Data (M2D): Shifting Complexity from GNNs to Graphs for Transparent Graph Learning

TL;DR

The paper introduces M2D distillation, a framework that transfers GNN model complexity into augmented graph data, making model behavior more transparent and interpretable without sacrificing performance.

Contribution

It presents a novel data-centric distillation method that enhances GNN transparency by embedding model complexity into graph data rather than the model itself.

Findings

M2D enables simple models to match complex GNN performance.

It reveals underlying mechanisms like fairness and attention in an interpretable manner.

M2D preserves performance while increasing transparency.

Abstract

Graph Neural Networks (GNNs) achieve high performance but can be opaque to humans, making it difficult to understand and compare the many proposed architectures. While existing explainability methods attribute individual predictions to nodes, edges, or features, they do not provide architectural transparency or explain the fundamental performance gap between simple and more complex models. To address this limitation, we introduce Model-to-Data (M2D) distillation, a new framework that increases transparency by transferring model complexity into the data space. M2D distills the teacher model into an augmented graph with enriched features and structure, enabling a simple student to match the teacher's performance. By materializing model behavior in the data, our approach allows humans to inspect architectural advantages directly. We show that M2D reveals underlying mechanisms such as fairness objectives and attention-based aggregation in an interpretable way, enhancing GNN transparency while preserving performance.