Resolve Highway Conflict in Multi-Autonomous Vehicle Controls with Local State Attention

TL;DR

This paper introduces a Local State Attention module for multi-agent reinforcement learning in autonomous driving, improving conflict resolution and merging efficiency in high-density highway scenarios.

Contribution

It proposes a novel attention-based input representation to better handle local conflicts and stochastic events in multi-autonomous vehicle control.

Findings

Significant improvement in merging efficiency over baselines.

Enhanced conflict resolution in high-density traffic.

Better generalization to stochastic events.

Abstract

In mixed-traffic environments, autonomous vehicles must adapt to human-controlled vehicles and other unusual driving situations. This setting can be framed as a multi-agent reinforcement learning (MARL) environment with full cooperative reward among the autonomous vehicles. While methods such as Multi-agent Proximal Policy Optimization can be effective in training MARL tasks, they often fail to resolve local conflict between agents and are unable to generalize to stochastic events. In this paper, we propose a Local State Attention module to assist the input state representation. By relying on the self-attention operator, the module is expected to compress the essential information of nearby agents to resolve the conflict in traffic situations. Utilizing a simulated highway merging scenario with the priority vehicle as the unexpected event, our approach is able to prioritize other vehicles' information to manage the merging process. The results demonstrate significant improvements in merging efficiency compared to popular baselines, especially in high-density traffic settings.