GNNAnatomy: Rethinking Model-Level Explanations for Graph Neural Networks

TL;DR

GNNAnatomy introduces a novel, interpretable approach for explaining GNNs by analyzing graphlet-based topologies with a surrogate model, addressing limitations of confidence-based explanations and supporting multi-grained, human-centric insights.

Contribution

The paper presents GNNAnatomy, a distillation-based explanation method that characterizes graph topologies with graphlets and enables multi-grained, human-interpretable explanations for GNNs.

Findings

Effective in identifying discriminative graph topologies

Supports multi-grained explanations for diverse graph subsets

Enhances transparency and trust in GNN explanations

Abstract

Graph Neural Networks (GNNs) achieve outstanding performance across graph-based tasks but remain difficult to interpret. In this paper, we revisit foundational assumptions underlying model-level explanation methods for GNNs, namely: (1) maximizing classification confidence yields representative explanations, (2) a single explanation suffices for an entire class of graphs, and (3) explanations are inherently trustworthy. We identify pitfalls resulting from these assumptions: methods that optimize for classification confidence may overlook partially learned patterns; topological diversity across graph subsets within the same class is often underrepresented; and explanations alone offer limited support for building user trust when applied to new datasets or models. This paper introduces GNNAnatomy, a distillation-based method designed to generate explanations while avoiding these pitfalls. GNNAnatomy first characterizes graph topology using graphlets, a set of fundamental substructures. We then train a transparent multilayer perceptron surrogate to directly approximate GNN predictions based on the graphlet representations. By analyzing the weights assigned to each graphlet, we identify the most discriminative topologies, which serve as GNN explanations. To account for structural diversity within a class, GNNAnatomy generates explanations at the required granularity through an interface that supports human-AI teaming. This interface helps users identify subsets of graphs where distinct critical substructures drive class differentiation, enabling multi-grained explanations. Additionally, by enabling exploration and linking explanations back to input graphs, the interface fosters greater transparency and trust. We evaluate GNNAnatomy on both synthetic and real-world datasets through quantitative metrics and qualitative comparisons with state-of-the-art model-level explainable GNN methods.