CAMAR: Continuous Actions Multi-Agent Routing

TL;DR

CAMAR introduces a new multi-agent reinforcement learning benchmark with continuous actions, supporting cooperative and competitive tasks, integrating classical planning methods, and enabling detailed performance analysis.

Contribution

This paper presents CAMAR, a novel MARL benchmark with continuous actions, a three-tier evaluation protocol, and integration of classical planning methods for enhanced research.

Findings

CAMAR is efficient at up to 100,000 steps per second.

Hybrid approaches combining RRT* with MARL algorithms outperform standalone methods.

CAMAR provides a challenging, realistic environment for MARL research.

Abstract

Multi-agent reinforcement learning (MARL) is a powerful paradigm for solving cooperative and competitive decision-making problems. While many MARL benchmarks have been proposed, few combine continuous state and action spaces with challenging coordination and planning tasks. We introduce CAMAR, a new MARL benchmark designed explicitly for multi-agent pathfinding in environments with continuous actions. CAMAR supports cooperative and competitive interactions between agents and runs efficiently at up to 100,000 environment steps per second. We also propose a three-tier evaluation protocol to better track algorithmic progress and enable deeper analysis of performance. In addition, CAMAR allows the integration of classical planning methods such as RRT and RRT* into MARL pipelines. We use them as standalone baselines and combine RRT* with popular MARL algorithms to create hybrid approaches. We provide a suite of test scenarios and benchmarking tools to ensure reproducibility and fair comparison. Experiments show that CAMAR presents a challenging and realistic testbed for the MARL community.