Faithful Explanations for Deep Graph Models

TL;DR

This paper introduces a formal framework for evaluating the faithfulness of explanations for GNNs, reveals limitations of existing methods, and proposes KEC, a new explanation technique that maximizes faithfulness by leveraging graph structure.

Contribution

It provides a general method for characterizing explanation faithfulness for GNNs, analyzes limitations of current methods, and proposes KEC, a novel explanation approach that enhances faithfulness.

Findings

Feature attribution methods miss nonlinear edge effects.

Existing subgraph explanations lack faithfulness.

KEC improves explanation faithfulness in experiments.

Abstract

This paper studies faithful explanations for Graph Neural Networks (GNNs). First, we provide a new and general method for formally characterizing the faithfulness of explanations for GNNs. It applies to existing explanation methods, including feature attributions and subgraph explanations. Second, our analytical and empirical results demonstrate that feature attribution methods cannot capture the nonlinear effect of edge features, while existing subgraph explanation methods are not faithful. Third, we introduce \emph{k-hop Explanation with a Convolutional Core} (KEC), a new explanation method that provably maximizes faithfulness to the original GNN by leveraging information about the graph structure in its adjacency matrix and its \emph{k-th} power. Lastly, our empirical results over both synthetic and real-world datasets for classification and anomaly detection tasks with GNNs demonstrate the effectiveness of our approach.