An Information-Theoretic Analysis of Temporal GNNs

TL;DR

This paper introduces an information-theoretic framework for analyzing Temporal Graph Neural Networks, utilizing concepts like the information bottleneck and a new Mutual Information Rate metric to better understand their behavior.

Contribution

It develops a formal information-theoretic framework for temporal GNNs, including a novel definition of Mutual Information Rate tailored for temporal analysis.

Findings

Proposes a new Mutual Information Rate metric for temporal GNNs

Adapts the information bottleneck concept for temporal network analysis

Provides a theoretical foundation for future analysis of temporal GNNs

Abstract

Temporal Graph Neural Networks, a new and trending area of machine learning, suffers from a lack of formal analysis. In this paper, information theory is used as the primary tool to provide a framework for the analysis of temporal GNNs. For this reason, the concept of information bottleneck is used and adjusted to be suitable for a temporal analysis of such networks. To this end, a new definition for Mutual Information Rate is provided, and the potential use of this new metric in the analysis of temporal GNNs is studied.