Zero-Shot MARL Benchmark in the Cyber-Physical Mobility Lab

TL;DR

This paper introduces a reproducible benchmark for evaluating zero-shot transfer of MARL policies for autonomous vehicles across simulation, digital twin, and real hardware, highlighting key challenges.

Contribution

It presents an open-source platform integrating simulation and real-world testing for systematic analysis of sim-to-real transfer in MARL for CAVs.

Findings

Identified architectural differences as a source of performance degradation.

Demonstrated performance gap increases with environmental realism.

Showcased the platform's utility for systematic analysis.

Abstract

We present a reproducible benchmark for evaluating sim-to-real transfer of Multi-Agent Reinforcement Learning (MARL) policies for Connected and Automated Vehicles (CAVs). The platform, based on the Cyber-Physical Mobility Lab (CPM Lab) [1], integrates simulation, a high-fidelity digital twin, and a physical testbed, enabling structured zero-shot evaluation of MARL motion-planning policies. We demonstrate its use by deploying a SigmaRL-trained policy [2] across all three domains, revealing two complementary sources of performance degradation: architectural differences between simulation and hardware control stacks, and the sim-to-real gap induced by increasing environmental realism. The open-source setup enables systematic analysis of sim-to-real challenges in MARL under realistic, reproducible conditions.