Logical Distillation of Graph Neural Networks

TL;DR

This paper introduces a logic-based, interpretable decision-tree model for graph learning that distills complex GNNs into succinct logical classifiers, maintaining accuracy and outperforming GNNs when the ground truth aligns with the logic.

Contribution

It presents a novel method to extract interpretable logical models from GNNs using an extension of C2 logic and decision trees, enhancing interpretability without sacrificing performance.

Findings

Distilled models are interpretable and succinct.

Distilled models achieve similar accuracy to GNNs.

Outperforms GNNs when the ground truth is in C2 logic.

Abstract

We present a logic based interpretable model for learning on graphs and an algorithm to distill this model from a Graph Neural Network (GNN). Recent results have shown connections between the expressivity of GNNs and the two-variable fragment of first-order logic with counting quantifiers (C2). We introduce a decision-tree based model which leverages an extension of C2 to distill interpretable logical classifiers from GNNs. We test our approach on multiple GNN architectures. The distilled models are interpretable, succinct, and attain similar accuracy to the underlying GNN. Furthermore, when the ground truth is expressible in C2, our approach outperforms the GNN.