Interpreting Temporal Graph Neural Networks with Koopman Theory

TL;DR

This paper introduces Koopman theory-inspired methods to interpret and explain the learned dynamics of spatiotemporal graph neural networks, enhancing understanding of their decision processes.

Contribution

The paper proposes two novel explainability techniques based on DMD and SINDy for temporal graph neural networks, applying Koopman theory for the first time in this context.

Findings

Methods correctly identify infection timings and nodes in semi-synthetic datasets.

Qualitative validation on human motion data highlights relevant body parts for action recognition.

Approaches provide interpretable insights into complex spatiotemporal dynamics.

Abstract

Spatiotemporal graph neural networks (STGNNs) have shown promising results in many domains, from forecasting to epidemiology. However, understanding the dynamics learned by these models and explaining their behaviour is significantly more difficult than for models that deal with static data. Inspired by Koopman theory, which allows a simple description of intricate, nonlinear dynamical systems, we introduce new explainability approaches for temporal graphs. Specifically, we present two methods to interpret the STGNN's decision process and identify the most relevant spatial and temporal patterns in the input for the task at hand. The first relies on dynamic mode decomposition (DMD), a Koopman-inspired dimensionality reduction method. The second relies on sparse identification of nonlinear dynamics (SINDy), a popular method for discovering governing equations of dynamical systems, which we use for the first time as a general tool for explainability. On semi-synthetic dissemination datasets, our methods correctly identify interpretable features such as the times at which infections occur and the infected nodes. We also validate the methods qualitatively on a real-world human motion dataset, where the explanations highlight the body parts most relevant for action recognition.