Scaling Up Dynamic Graph Representation Learning via Spiking Neural Networks

TL;DR

SpikeNet introduces a scalable approach using spiking neural networks to model dynamic graphs efficiently, outperforming traditional RNN-based methods in large-scale temporal graph tasks with lower computational costs.

Contribution

The paper proposes SpikeNet, a novel framework leveraging spiking neural networks for scalable and efficient dynamic graph representation learning on large temporal graphs.

Findings

SpikeNet outperforms strong baselines on temporal node classification.

It generalizes well to large graphs with fewer parameters.

It achieves lower computational costs compared to RNN-based methods.

Abstract

Recent years have seen a surge in research on dynamic graph representation learning, which aims to model temporal graphs that are dynamic and evolving constantly over time. However, current work typically models graph dynamics with recurrent neural networks (RNNs), making them suffer seriously from computation and memory overheads on large temporal graphs. So far, scalability of dynamic graph representation learning on large temporal graphs remains one of the major challenges. In this paper, we present a scalable framework, namely SpikeNet, to efficiently capture the temporal and structural patterns of temporal graphs. We explore a new direction in that we can capture the evolving dynamics of temporal graphs with spiking neural networks (SNNs) instead of RNNs. As a low-power alternative to RNNs, SNNs explicitly model graph dynamics as spike trains of neuron populations and enable spike-based propagation in an efficient way. Experiments on three large real-world temporal graph datasets demonstrate that SpikeNet outperforms strong baselines on the temporal node classification task with lower computational costs. Particularly, SpikeNet generalizes to a large temporal graph (2.7M nodes and 13.9M edges) with significantly fewer parameters and computation overheads.Our code is publicly available at \url{https://github.com/EdisonLeeeee/SpikeNet}.