A Multi-Layer CNN-GRUSKIP model based on transformer for spatial TEMPORAL traffic flow prediction

TL;DR

This paper introduces a novel multi-layer CNN-GRUSKIP transformer-based model that effectively captures complex spatial-temporal traffic patterns, outperforming existing models in real-world traffic flow prediction tasks.

Contribution

It presents a new hybrid CNN-GRU-SKIP architecture combined with transformer modules, enhancing long-term dependency modeling and prediction accuracy in traffic flow forecasting.

Findings

Outperforms ARIMA, Graph Wave Net, HA, LSTM, STGCN, APTN on real datasets

Effectively captures extended and erratic traffic patterns

Demonstrates superior predictive accuracy in California traffic data

Abstract

Traffic flow prediction remains a cornerstone for intelligent transportation systems ITS, influencing both route optimization and environmental efforts. While Recurrent Neural Networks RNN and traditional Convolutional Neural Networks CNN offer some insights into the spatial temporal dynamics of traffic data, they are often limited when navigating sparse and extended spatial temporal patterns. In response, the CNN-GRUSKIP model emerges as a pioneering approach. Notably, it integrates the GRU-SKIP mechanism, a hybrid model that leverages the Gate Recurrent Unit of GRU capabilities to process sequences with the SKIP feature of ability to bypass and connect longer temporal dependencies, making it especially potent for traffic flow predictions with erratic and extended patterns. Another distinctive aspect is its non-standard 6-layer CNN, meticulously designed for in-depth spatiotemporal correlation extraction. The model comprises (1) the specialized CNN feature extraction, (2) the GRU-SKIP enhanced long-temporal module adept at capturing extended patterns, (3) a transformer module employing encoder-decoder and multi-attention mechanisms to hone prediction accuracy and trim model complexity, and (4) a bespoke prediction module. When tested against real-world datasets from California of Caltrans Performance Measurement System PeMS, specifically PeMS districts 4 and 8, the CNN-GRUSKIP consistently outperformed established models such as ARIMA, Graph Wave Net, HA, LSTM, STGCN, and APTN. With its potent predictive prowess and adaptive architecture, the CNN-GRUSKIP model stands to redefine ITS applications, especially where nuanced traffic dynamics are in play.