Curriculum Negative Mining For Temporal Networks

TL;DR

This paper introduces Curriculum Negative Mining (CurNM), a novel negative sampling framework for training Temporal Graph Neural Networks, addressing positive sparsity and shift challenges to improve model robustness and performance.

Contribution

We propose a model-aware curriculum learning framework with dynamic negative pools and temporal-aware selection to enhance negative sampling in TGNN training.

Findings

Outperforms baseline methods on 12 datasets and 3 TGNN models.

Improves robustness and stability of training through curriculum learning.

Ablation studies confirm effectiveness and parameter sensitivity analysis shows robustness.

Abstract

Temporal networks are effective in capturing the evolving interactions of networks over time, such as social networks and e-commerce networks. In recent years, researchers have primarily concentrated on developing specific model architectures for Temporal Graph Neural Networks (TGNNs) in order to improve the representation quality of temporal nodes and edges. However, limited attention has been given to the quality of negative samples during the training of TGNNs. When compared with static networks, temporal networks present two specific challenges for negative sampling: positive sparsity and positive shift. Positive sparsity refers to the presence of a single positive sample amidst numerous negative samples at each timestamp, while positive shift relates to the variations in positive samples across different timestamps. To robustly address these challenges in training TGNNs, we introduce Curriculum Negative Mining (CurNM), a model-aware curriculum learning framework that adaptively adjusts the difficulty of negative samples. Within this framework, we first establish a dynamically updated negative pool that balances random, historical, and hard negatives to address the challenges posed by positive sparsity. Secondly, we implement a temporal-aware negative selection module that focuses on learning from the disentangled factors of recently active edges, thus accurately capturing shifting preferences. Finally, the selected negatives are combined with annealing random negatives to support stable training. Extensive experiments on 12 datasets and 3 TGNNs demonstrate that our method outperforms baseline methods by a significant margin. Additionally, thorough ablation studies and parameter sensitivity experiments verify the usefulness and robustness of our approach.