A Dynamic Holding Approach to Stabilizing a Bus Line Based on the Q-learning Algorithm with Multistage Look-ahead

TL;DR

This paper introduces an adaptive bus holding control method combining approximate dynamic programming with multi-stage look-ahead and neural networks, effectively stabilizing bus headways and reducing bunching in high-frequency bus lines.

Contribution

It presents a novel dynamic holding strategy integrating multi-stage look-ahead with Q-learning and neural networks for real-time bus operation stabilization.

Findings

Effectively stabilizes bus headways and reduces bunching.

Produces more evenly distributed headways compared to traditional strategies.

Shortens passenger waiting times significantly.

Abstract

The unreliable service and the unstable operation of a high frequency bus line are shown as bus bunching and the uneven distribution of headways along the bus line. Although many control strategies, such as the static and dynamic holding strategies, have been proposed to solve the above problems, many of them take on some oversimplified assumptions about the real bus line operation. So it is hard for them to continuously adapt to the evolving complex system. In view of this dynamic setting, we present an adaptive holding method which combines the classic approximate dynamic programming (ADP) with the multi-stage look-ahead mechanism. The holding time, that is the only control means used in this study, will be determined by estimating its impact on the operation stability of the bus line system in the remained observation period. The multi-stage look-ahead mechanism introduced into the classic Q-learning algorithm of the ADP model makes the algorithm get through its earlier unstable phase more quickly and easily. During the implementation of the new holding approach, the past experiences of holding operations can be cumulated effectively into an artificial neural network used to approximate the unavailable Q-factor. The use of a detailed simulation system in the new approach makes it possible to take into accounts most of the possible causes of instability. The numerical experiments show that the new holding approach can stabilize the system by producing evenly distributed headway and removing bus bunching thoroughly. Comparing with the terminal station holding strategies, the new method brings a more reliable bus line with shorter waiting times for passengers.