CPDG: A Contrastive Pre-Training Method for Dynamic Graph Neural Networks

TL;DR

This paper introduces CPDG, a contrastive pre-training approach for dynamic graph neural networks that improves generalization and temporal modeling, demonstrating superior performance on large-scale datasets.

Contribution

The paper proposes a novel contrastive pre-training method for DGNNs, addressing generalization and temporal modeling challenges with a structural-temporal sampler and contrastive schemes.

Findings

CPDG outperforms existing methods on research datasets.

CPDG improves performance across various downstream tasks.

CPDG demonstrates effectiveness in industrial dynamic graph scenarios.

Abstract

Dynamic graph data mining has gained popularity in recent years due to the rich information contained in dynamic graphs and their widespread use in the real world. Despite the advances in dynamic graph neural networks (DGNNs), the rich information and diverse downstream tasks have posed significant difficulties for the practical application of DGNNs in industrial scenarios. To this end, in this paper, we propose to address them by pre-training and present the Contrastive Pre-Training Method for Dynamic Graph Neural Networks (CPDG). CPDG tackles the challenges of pre-training for DGNNs, including generalization capability and long-short term modeling capability, through a flexible structural-temporal subgraph sampler along with structural-temporal contrastive pre-training schemes. Extensive experiments conducted on both large-scale research and industrial dynamic graph datasets show that CPDG outperforms existing methods in dynamic graph pre-training for various downstream tasks under three transfer settings.