Traffic Signal Control with Communicative Deep Reinforcement Learning Agents: a Case Study

TL;DR

This paper evaluates the effectiveness of the Multi-Agent Advantage Actor-Critic (MA2C) algorithm for adaptive traffic signal control, demonstrating its potential to outperform other methods through theoretical analysis and simulation in Bologna.

Contribution

It provides a theoretical framework for MA2C using non-Markov decision processes and empirically tests its robustness and performance in real-world traffic simulations.

Findings

MA2C outperforms alternative algorithms in simulated traffic scenarios.

Theoretical analysis offers deeper insights into MA2C's decision-making process.

MA2C trained with pseudo-random vehicle flows shows promising results.

Abstract

In this work we analyze Multi-Agent Advantage Actor-Critic (MA2C) a recently proposed multi-agent reinforcement learning algorithm that can be applied to adaptive traffic signal control (ATSC) problems. To evaluate its potential we compare MA2C with Independent Advantage Actor-Critic (IA2C) and other Reinforcement Learning or heuristic based algorithms. Specifically, we analyze MA2C theoretically with the framework provided by non-Markov decision processes, which allows a deeper insight of the algorithm, and we critically examine the effectiveness and the robustness of the method by testing it in two traffic areas located in Bologna (Italy) simulated in SUMO, a software modeling tool for ATSC problems. Our results indicate that MA2C, trained with pseudo-random vehicle flows, is a promising technique able to outperform the alternative methods.