Graph Retention Networks for Dynamic Graphs

TL;DR

The paper introduces Graph Retention Networks (GRNs), a novel architecture for efficient, scalable deep learning on dynamic graphs, achieving state-of-the-art performance with reduced latency and memory usage.

Contribution

It presents a unified, efficient architecture for dynamic graph learning that balances effectiveness, scalability, and computational cost, with extensive experimental validation.

Findings

Significantly reduces training latency and GPU memory overhead.

Improves inference throughput by up to 86.7x over SOTA methods.

Achieves competitive performance across various dynamic graph benchmarks.

Abstract

In this paper, we propose Graph Retention Networks (GRNs) as a unified architecture for deep learning on dynamic graphs. The GRN extends the concept of retention into dynamic graph data as graph retention, equipping the model with three key computational paradigms: parallelizable training, low-cost $\mathcal{O}(1)$ inference, and long-term chunkwise training. This architecture achieves an optimal balance between efficiency, effectiveness, and scalability. Extensive experiments on benchmark datasets demonstrate its strong performance in both edge-level prediction and node-level classification tasks with significantly reduced training latency, lower GPU memory overhead, and improved inference throughput by up to 86.7x compared to SOTA baselines. The proposed GRN architecture achieves competitive performance across diverse dynamic graph benchmarks, demonstrating its adaptability to a wide range of tasks.