Beyond Topological Self-Explainable GNNs: A Formal Explainability Perspective

TL;DR

This paper formalizes explanations in self-explainable GNNs, compares them to established explanation notions, and proposes Dual-Channel GNNs to improve explainability and performance.

Contribution

It introduces a formal analysis of explanations in SE-GNNs, compares them with PI and faithful explanations, and proposes Dual-Channel GNNs combining rule extraction with SE-GNNs.

Findings

Minimal Explanations match Prime Implicant explanations in some tasks.

SE-GNN explanations can be less informative and less faithful than PI explanations.

Dual-Channel GNNs can recover succinct rules and match or outperform standard SE-GNNs.

Abstract

Self-Explainable Graph Neural Networks (SE-GNNs) are popular explainable-by-design GNNs, but their explanations' properties and limitations are not well understood. Our first contribution fills this gap by formalizing the explanations extracted by some popular SE-GNNs, referred to as Minimal Explanations (MEs), and comparing them to established notions of explanations, namely Prime Implicant (PI) and faithful explanations. Our analysis reveals that MEs match PI explanations for a restricted but significant family of tasks. In general, however, they can be less informative than PI explanations and are surprisingly misaligned with widely accepted notions of faithfulness. Although faithful and PI explanations are informative, they are intractable to find and we show that they can be prohibitively large. Given these observations, a natural choice is to augment SE-GNNs with alternative modalities of explanations taking care of SE-GNNs' limitations. To this end, we propose Dual-Channel GNNs that integrate a white-box rule extractor and a standard SE-GNN, adaptively combining both channels. Our experiments show that even a simple instantiation of Dual-Channel GNNs can recover succinct rules and perform on par or better than widely used SE-GNNs.