OpenCity: Open Spatio-Temporal Foundation Models for Traffic Prediction

TL;DR

OpenCity is a novel spatio-temporal foundation model for traffic prediction that leverages Transformer and graph neural networks to achieve zero-shot generalization across diverse urban environments, demonstrating strong scalability and adaptability.

Contribution

This work introduces OpenCity, a pre-trained foundation model combining Transformers and graph neural networks for versatile, zero-shot traffic forecasting across various cities.

Findings

OpenCity achieves exceptional zero-shot prediction performance.

The model demonstrates promising scaling laws for adaptability.

OpenCity is open-source for community use and further development.

Abstract

Accurate traffic forecasting is crucial for effective urban planning and transportation management, enabling efficient resource allocation and enhanced travel experiences. However, existing models often face limitations in generalization, struggling with zero-shot prediction on unseen regions and cities, as well as diminished long-term accuracy. This is primarily due to the inherent challenges in handling the spatial and temporal heterogeneity of traffic data, coupled with the significant distribution shift across time and space. In this work, we aim to unlock new possibilities for building versatile, resilient and adaptive spatio-temporal foundation models for traffic prediction. To achieve this goal, we introduce a novel foundation model, named OpenCity, that can effectively capture and normalize the underlying spatio-temporal patterns from diverse data characteristics, facilitating zero-shot generalization across diverse urban environments. OpenCity integrates the Transformer architecture with graph neural networks to model the complex spatio-temporal dependencies in traffic data. By pre-training OpenCity on large-scale, heterogeneous traffic datasets, we enable the model to learn rich, generalizable representations that can be seamlessly applied to a wide range of traffic forecasting scenarios. Experimental results demonstrate that OpenCity exhibits exceptional zero-shot predictive performance. Moreover, OpenCity showcases promising scaling laws, suggesting the potential for developing a truly one-for-all traffic prediction solution that can adapt to new urban contexts with minimal overhead. We made our proposed OpenCity model open-source and it is available at the following link: https://github.com/HKUDS/OpenCity.