STGformer: Efficient Spatiotemporal Graph Transformer for Traffic Forecasting

TL;DR

STGformer is a novel spatiotemporal graph transformer that significantly improves traffic forecasting efficiency by reducing computational costs and memory usage while maintaining high accuracy, enabling real-world large-scale applications.

Contribution

The paper introduces STGformer, a new architecture that captures high-order spatiotemporal interactions in a single layer, achieving 100x speedup and 99.8% memory reduction over previous models.

Findings

Achieves 100x faster inference than STAEformer.

Reduces GPU memory usage by 99.8%.

Outperforms state-of-the-art methods on LargeST benchmark.

Abstract

Traffic forecasting is a cornerstone of smart city management, enabling efficient resource allocation and transportation planning. Deep learning, with its ability to capture complex nonlinear patterns in spatiotemporal (ST) data, has emerged as a powerful tool for traffic forecasting. While graph neural networks (GCNs) and transformer-based models have shown promise, their computational demands often hinder their application to real-world road networks, particularly those with large-scale spatiotemporal interactions. To address these challenges, we propose a novel spatiotemporal graph transformer (STGformer) architecture. STGformer effectively balances the strengths of GCNs and Transformers, enabling efficient modeling of both global and local traffic patterns while maintaining a manageable computational footprint. Unlike traditional approaches that require multiple attention layers, STG attention block captures high-order spatiotemporal interactions in a single layer, significantly reducing computational cost. In particular, STGformer achieves a 100x speedup and a 99.8\% reduction in GPU memory usage compared to STAEformer during batch inference on a California road graph with 8,600 sensors. We evaluate STGformer on the LargeST benchmark and demonstrate its superiority over state-of-the-art Transformer-based methods such as PDFormer and STAEformer, which underline STGformer's potential to revolutionize traffic forecasting by overcoming the computational and memory limitations of existing approaches, making it a promising foundation for future spatiotemporal modeling tasks.