XGNN: Towards Model-Level Explanations of Graph Neural Networks

TL;DR

XGNN introduces a model-level explanation method for GNNs by training a graph generator via reinforcement learning to produce graph patterns that reveal how GNNs make predictions, aiding interpretability and improvement.

Contribution

The paper presents a novel reinforcement learning-based graph generator to interpret GNNs at the model level, providing high-level insights and explanations.

Findings

Generated graphs help understand GNN decision processes

Method verifies GNN behavior on synthetic and real data

Guides improvements in GNN performance

Abstract

Graphs neural networks (GNNs) learn node features by aggregating and combining neighbor information, which have achieved promising performance on many graph tasks. However, GNNs are mostly treated as black-boxes and lack human intelligible explanations. Thus, they cannot be fully trusted and used in certain application domains if GNN models cannot be explained. In this work, we propose a novel approach, known as XGNN, to interpret GNNs at the model-level. Our approach can provide high-level insights and generic understanding of how GNNs work. In particular, we propose to explain GNNs by training a graph generator so that the generated graph patterns maximize a certain prediction of the model.We formulate the graph generation as a reinforcement learning task, where for each step, the graph generator predicts how to add an edge into the current graph. The graph generator is trained via a policy gradient method based on information from the trained GNNs. In addition, we incorporate several graph rules to encourage the generated graphs to be valid. Experimental results on both synthetic and real-world datasets show that our proposed methods help understand and verify the trained GNNs. Furthermore, our experimental results indicate that the generated graphs can provide guidance on how to improve the trained GNNs.