OPTIMA: Optimized Policy for Intelligent Multi-Agent Systems Enables Coordination-Aware Autonomous Vehicles

TL;DR

OPTIMA is a distributed reinforcement learning framework designed to enhance coordination, safety, and efficiency of autonomous vehicles in complex environments by extensive environment exploration and adaptable training systems.

Contribution

It introduces a novel distributed RL approach for CAV cooperation that balances thorough environment exploration with efficient policy optimization.

Findings

Improved coordination among autonomous vehicles in complex scenarios.

Enhanced safety and efficiency in multi-agent autonomous driving.

Flexible training system adaptable to various algorithms and strategies.

Abstract

Coordination among connected and autonomous vehicles (CAVs) is advancing due to developments in control and communication technologies. However, much of the current work is based on oversimplified and unrealistic task-specific assumptions, which may introduce vulnerabilities. This is critical because CAVs not only interact with their environment but are also integral parts of it. Insufficient exploration can result in policies that carry latent risks, highlighting the need for methods that explore the environment both extensively and efficiently. This work introduces OPTIMA, a novel distributed reinforcement learning framework for cooperative autonomous vehicle tasks. OPTIMA alternates between thorough data sampling from environmental interactions and multi-agent reinforcement learning algorithms to optimize CAV cooperation, emphasizing both safety and efficiency. Our goal is to improve the generality and performance of CAVs in highly complex and crowded scenarios. Furthermore, the industrial-scale distributed training system easily adapts to different algorithms, reward functions, and strategies.