MARLens: Understanding Multi-agent Reinforcement Learning for Traffic Signal Control via Visual Analytics

TL;DR

MARLens is a visual analytics tool designed to interpret multi-agent reinforcement learning models for traffic signal control, addressing the challenge of understanding complex decision-making processes in multi-intersection traffic management.

Contribution

The paper introduces MARLens, a novel visual analytics system that enhances interpretability of MARL models in traffic signal control, supporting exploration of decision processes and agent interactions.

Findings

MARLens effectively reveals model decision-making processes.

The system aids in identifying critical traffic states quickly.

Case studies demonstrate improved understanding of MARL in TSC.

Abstract

The issue of traffic congestion poses a significant obstacle to the development of global cities. One promising solution to tackle this problem is intelligent traffic signal control (TSC). Recently, TSC strategies leveraging reinforcement learning (RL) have garnered attention among researchers. However, the evaluation of these models has primarily relied on fixed metrics like reward and queue length. This limited evaluation approach provides only a narrow view of the model's decision-making process, impeding its practical implementation. Moreover, effective TSC necessitates coordinated actions across multiple intersections. Existing visual analysis solutions fall short when applied in multi-agent settings. In this study, we delve into the challenge of interpretability in multi-agent reinforcement learning (MARL), particularly within the context of TSC. We propose MARLens a visual analytics system tailored to understand MARL-based TSC. Our system serves as a versatile platform for both RL and TSC researchers. It empowers them to explore the model's features from various perspectives, revealing its decision-making processes and shedding light on interactions among different agents. To facilitate quick identification of critical states, we have devised multiple visualization views, complemented by a traffic simulation module that allows users to replay specific training scenarios. To validate the utility of our proposed system, we present three comprehensive case studies, incorporate insights from domain experts through interviews, and conduct a user study. These collective efforts underscore the feasibility and effectiveness of MARLens in enhancing our understanding of MARL-based TSC systems and pave the way for more informed and efficient traffic management strategies.