Informative Subgraphs Aware Masked Auto-Encoder in Dynamic Graphs

TL;DR

This paper introduces DyGIS, a novel masked autoencoder for dynamic graphs that generates informative subgraphs to preserve spatio-temporal evolution information, leading to improved performance in various tasks.

Contribution

DyGIS is the first method to incorporate informative subgraph generation into masked autoencoders for dynamic graphs, addressing the loss of evolution information.

Findings

DyGIS outperforms existing methods on eleven datasets.

It effectively preserves dynamic evolution information.

Achieves state-of-the-art results across multiple tasks.

Abstract

Generative self-supervised learning (SSL), especially masked autoencoders (MAE), has greatly succeeded and garnered substantial research interest in graph machine learning. However, the research of MAE in dynamic graphs is still scant. This gap is primarily due to the dynamic graph not only possessing topological structure information but also encapsulating temporal evolution dependency. Applying a random masking strategy which most MAE methods adopt to dynamic graphs will remove the crucial subgraph that guides the evolution of dynamic graphs, resulting in the loss of crucial spatio-temporal information in node representations. To bridge this gap, in this paper, we propose a novel Informative Subgraphs Aware Masked Auto-Encoder in Dynamic Graph, namely DyGIS. Specifically, we introduce a constrained probabilistic generative model to generate informative subgraphs that guide the evolution of dynamic graphs, successfully alleviating the issue of missing dynamic evolution subgraphs. The informative subgraph identified by DyGIS will serve as the input of dynamic graph masked autoencoder (DGMAE), effectively ensuring the integrity of the evolutionary spatio-temporal information within dynamic graphs. Extensive experiments on eleven datasets demonstrate that DyGIS achieves state-of-the-art performance across multiple tasks.