Explaining Temporal Graph Predictions With Shapley Values

TL;DR

This paper introduces two novel, model-agnostic Shapley-based explainers for Temporal Graph Neural Networks, providing interpretable insights into model behavior and uncovering issues in existing implementations.

Contribution

It proposes event-level and feature-level Shapley explainers for TGNNs, extending interpretability and revealing model biases and implementation flaws.

Findings

Explainability methods outperform state-of-the-art on multiple datasets.

Feature Explainer uncovers faulty timestamp extraction in TGAT.

Methods provide hierarchical insights into model decision processes.

Abstract

Temporal Graph Neural Networks (TGNNs) have become increasingly popular in recent years due to their superior predictive performance by combining both spatial and temporal information. However, how these models utilize the information to make predictions is rather unexplored, leading to potentially faulty or biased models. This work introduces two novel model-agnostic explainers for local explanations of TGNNs based on Shapley and Owen values. The first method, an event-level (edge-level) Shapley explainer, applies the KernelSHAP algorithm to estimate contribution scores for individual temporal events, providing interpretable descriptions for model behavior. The second, a feature-level Shapley explainer, extends this framework by decomposing event-level Shapley values into Owen values, and thereby uncovers hierarchical dependencies of the event and its features. The explainers outperform SOTA explainers on different metrics and datasets. Additionally, the Feature Explainer reveals a faulty extraction of actual timestamps of a commonly used TGAT implementation, helping to further understand performance drops on very sparse explanations.