A Hysteretic Q-learning Coordination Framework for Emerging Mobility Systems in Smart Cities

TL;DR

This paper proposes a decentralized Q-learning based coordination framework for connected autonomous vehicles at intersections, aiming to reduce travel time and fuel consumption in smart city traffic management.

Contribution

It introduces a novel hysteretic Q-learning coordination mechanism combined with FIFO policy for signal-free intersection management in smart cities.

Findings

The approach reduces travel time compared to classical methods.

It improves fuel efficiency in simulated scenarios.

Demonstrates effectiveness through simulation comparisons.

Abstract

Connected and automated vehicles (CAVs) can alleviate traffic congestion, air pollution, and improve safety. In this paper, we provide a decentralized coordination framework for CAVs at a signal-free intersection to minimize travel time and improve fuel efficiency. We employ a simple yet powerful reinforcement learning approach, an off-policy temporal difference learning called Q-learning, enhanced with a coordination mechanism to address this problem. Then, we integrate a first-in-first-out queuing policy to improve the performance of our system. We demonstrate the efficacy of our proposed approach through simulation and comparison with the classical optimal control method based on Pontryagin's minimum principle.