Enhancing Traffic Prediction with Textual Data Using Large Language Models

TL;DR

This paper introduces a novel method that leverages large language models to process textual data, generating embeddings that enhance traditional traffic prediction models, especially under exceptional circumstances and with contextual information.

Contribution

It proposes a hybrid approach combining large language model embeddings with traditional models to improve short-term traffic prediction accuracy.

Findings

Significant accuracy improvement on New York Bike dataset

Effective integration of textual data into traffic forecasting models

Enhanced prediction under exceptional scenarios

Abstract

Traffic prediction is pivotal for rational transportation supply scheduling and allocation. Existing researches into short-term traffic prediction, however, face challenges in adequately addressing exceptional circumstances and integrating non-numerical contextual information like weather into models. While, Large language models offer a promising solution due to their inherent world knowledge. However, directly using them for traffic prediction presents drawbacks such as high cost, lack of determinism, and limited mathematical capability. To mitigate these issues, this study proposes a novel approach. Instead of directly employing large models for prediction, it utilizes them to process textual information and obtain embeddings. These embeddings are then combined with historical traffic data and inputted into traditional spatiotemporal forecasting models. The study investigates two types of special scenarios: regional-level and node-level. For regional-level scenarios, textual information is represented as a node connected to the entire network. For node-level scenarios, embeddings from the large model represent additional nodes connected only to corresponding nodes. This approach shows a significant improvement in prediction accuracy according to our experiment of New York Bike dataset.