Integrating Conventional Headway Control with Reinforcement Learning to Avoid Bus Bunching

TL;DR

This paper introduces IPPO-DH, a hybrid control model combining conventional headway control with reinforcement learning to improve bus bunching prevention, balancing efficiency and stability in transit systems.

Contribution

The paper presents a novel integrated proximal policy optimization model with dual-headway that combines traditional and RL methods for bus headway control.

Findings

IPPO-DH outperforms pure RL in stability.

IPPO-DH improves efficiency over conventional methods.

The model effectively balances stability and efficiency in simulations.

Abstract

Bus bunching is a natural-occurring phenomenon that undermines the efficiency and stability of the public transportation system. The mainstream solutions control the bus to intentionally stay longer at certain stations. Existing control methods include conventional methods that provide a formula to calculate the control time and reinforcement learning (RL) methods that determine the control policy through repeated interactions with the system. In this paper, we propose an integrated proximal policy optimization model with dual-headway (IPPO-DH). IPPO-DH integrates the conventional headway control with reinforcement learning, so that it acquires the advantages of both algorithms -- it is more efficient in normal environments and more stable in harsh ones. To demonstrate such an advantage, we design a bus simulation environment and compare IPPO-DH with RL and several conventional methods. The results show that the proposed model maintains the application value of the conventional method by avoiding the instability of the RL method in certain environments, and improves the efficiency compared with the conventional control, shedding new light on real-world bus transit system optimization.