Traffic Co-Simulation Framework Empowered by Infrastructure Camera Sensing and Reinforcement Learning

TL;DR

This paper introduces a co-simulation framework combining high-fidelity 3D modeling and large-scale traffic simulation, utilizing camera-based vehicle detection and MARL to optimize traffic signals in realistic urban scenarios.

Contribution

It presents a novel integrated framework using CARLA and SUMO with vision-based vehicle detection for reinforcement learning in traffic control, addressing real-world sensing limitations.

Findings

MARL with visual feedback improves traffic flow.

Better detection accuracy enhances control performance.

Robustness of MARL under sensing faults demonstrated.

Abstract

Traffic simulations are commonly used to optimize urban traffic flow, with reinforcement learning (RL) showing promising potential for automated traffic signal control, particularly in intelligent transportation systems involving connected automated vehicles. Multi-agent reinforcement learning (MARL) is particularly effective for learning control strategies for traffic lights in a network using iterative simulations. However, existing methods often assume perfect vehicle detection, which overlooks real-world limitations related to infrastructure availability and sensor reliability. This study proposes a co-simulation framework integrating CARLA and SUMO, which combines high-fidelity 3D modeling with large-scale traffic flow simulation. Cameras mounted on traffic light poles within the CARLA environment use a YOLO-based computer vision system to detect and count vehicles, providing real-time traffic data as input for adaptive signal control in SUMO. MARL agents trained with four different reward structures leverage this visual feedback to optimize signal timings and improve network-wide traffic flow. Experiments in a multi-intersection test-bed demonstrate the effectiveness of the proposed MARL approach in enhancing traffic conditions using real-time camera based detection. The framework also evaluates the robustness of MARL under faulty or sparse sensing and compares the performance of YOLOv5 and YOLOv8 for vehicle detection. Results show that while better accuracy improves performance, MARL agents can still achieve significant improvements with imperfect detection, demonstrating scalability and adaptability for real-world scenarios.