SwinUNet3D -- A Hierarchical Architecture for Deep Traffic Prediction using Shifted Window Transformers

TL;DR

This paper introduces SwinUNet3D, a novel hierarchical vision transformer architecture for deep traffic prediction that replaces convolutional blocks with shifted window transformers, demonstrating effectiveness on traffic forecasting data.

Contribution

The paper proposes a convolution-free UNet architecture using 3D shifted window transformers for spatiotemporal traffic prediction, enhancing modeling of complex traffic dynamics.

Findings

Achieved competitive results on Traffic4cast2021 dataset.

Demonstrated the effectiveness of transformer-based architecture in traffic forecasting.

Provided open-source code for reproducibility.

Abstract

Traffic forecasting is an important element of mobility management, an important key that drives the logistics industry. Over the years, lots of work have been done in Traffic forecasting using time series as well as spatiotemporal dynamic forecasting. In this paper, we explore the use of vision transformer in a UNet setting. We completely remove all convolution-based building blocks in UNet, while using 3D shifted window transformer in both encoder and decoder branches. In addition, we experiment with the use of feature mixing just before patch encoding to control the inter-relationship of the feature while avoiding contraction of the depth dimension of our spatiotemporal input. The proposed network is tested on the data provided by Traffic Map Movie Forecasting Challenge 2021(Traffic4cast2021), held in the competition track of Neural Information Processing Systems (NeurIPS). Traffic4cast2021 task is to predict an hour (6 frames) of traffic conditions (volume and average speed)from one hour of given traffic state (12 frames averaged in 5 minutes time span). Source code is available online at https://github.com/bojesomo/Traffic4Cast2021-SwinUNet3D.