ConvDySAT: Deep Neural Representation Learning on Dynamic Graphs via Self-Attention and Convolutional Neural Networks

TL;DR

ConvDySAT introduces a novel neural network model combining self-attention and convolutional layers to improve dynamic graph node representation learning, outperforming existing methods in link prediction tasks.

Contribution

It enhances DySAT by integrating convolutional neural networks with self-attention, capturing structural and temporal evolution more effectively in dynamic graphs.

Findings

Significant performance improvements over state-of-the-art methods.

Effective in single-step link prediction on communication and rating networks.

Demonstrates robustness in modeling evolving graph structures.

Abstract

Learning node representations on temporal graphs is a fundamental step to learn real-word dynamic graphs efficiently. Real-world graphs have the nature of continuously evolving over time, such as changing edges weights, removing and adding nodes and appearing and disappearing of edges, while previous graph representation learning methods focused generally on static graphs. We present ConvDySAT as an enhancement of DySAT, one of the state-of-the-art dynamic methods, by augmenting convolution neural networks with the self-attention mechanism, the employed method in DySAT to express the structural and temporal evolution. We conducted single-step link prediction on a communication network and rating network, Experimental results show significant performance gains for ConvDySAT over various state-of-the-art methods.