DALC: Distributed Automatic LSTM Customization for Fine-Grained Traffic Speed Prediction

TL;DR

This paper introduces DALC, a distributed method for automatically customizing LSTM models for fine-grained, large-scale traffic speed prediction, significantly improving accuracy over existing approaches.

Contribution

It formulates LSTM customization as a Markov decision process and proposes DALC, a distributed algorithm for scalable, accurate traffic prediction in large networks.

Findings

DALC outperforms Apache Spark MLlib in prediction accuracy.

The approach reduces customization time for large-scale networks.

Distributed customization enables scalable, fine-grained traffic prediction.

Abstract

Over the past decade, several approaches have been introduced for short-term traffic prediction. However, providing fine-grained traffic prediction for large-scale transportation networks where numerous detectors are geographically deployed to collect traffic data is still an open issue. To address this issue, in this paper, we formulate the problem of customizing an LSTM model for a single detector into a finite Markov decision process and then introduce an Automatic LSTM Customization (ALC) algorithm to automatically customize an LSTM model for a single detector such that the corresponding prediction accuracy can be as satisfactory as possible and the time consumption can be as low as possible. Based on the ALC algorithm, we introduce a distributed approach called Distributed Automatic LSTM Customization (DALC) to customize an LSTM model for every detector in large-scale transportation networks. Our experiment demonstrates that the DALC provides higher prediction accuracy than several approaches provided by Apache Spark MLlib.