Learning to Change: Choreographing Mixed Traffic Through Lateral Control and Hierarchical Reinforcement Learning

TL;DR

This paper presents a hierarchical reinforcement learning framework for managing mixed traffic at intersections, effectively coordinating robot and human-driven vehicles to reduce waiting times and improve traffic flow.

Contribution

It introduces a novel hierarchical RL approach that combines high-level decision making with precise vehicle control for mixed traffic management.

Findings

Reduces average waiting time by up to 54% compared to existing methods.

Outperforms traditional traffic signals when RV penetration exceeds 60%.

Demonstrates robustness and efficiency in complex intersection scenarios.

Abstract

The management of mixed traffic that consists of robot vehicles (RVs) and human-driven vehicles (HVs) at complex intersections presents a multifaceted challenge. Traditional signal controls often struggle to adapt to dynamic traffic conditions and heterogeneous vehicle types. Recent advancements have turned to strategies based on reinforcement learning (RL), leveraging its model-free nature, real-time operation, and generalizability over different scenarios. We introduce a hierarchical RL framework to manage mixed traffic through precise longitudinal and lateral control of RVs. Our proposed hierarchical framework combines the state-of-the-art mixed traffic control algorithm as a high level decision maker to improve the performance and robustness of the whole system. Our experiments demonstrate that the framework can reduce the average waiting time by up to 54% compared to the state-of-the-art mixed traffic control method. When the RV penetration rate exceeds 60%, our technique consistently outperforms conventional traffic signal control programs in terms of the average waiting time for all vehicles at the intersection.