Cooperative Decision-Making for CAVs at Unsignalized Intersections: A MARL Approach with Attention and Hierarchical Game Priors

TL;DR

This paper introduces MA-GA-DDPG, a multi-agent reinforcement learning algorithm with attention and hierarchical game priors, to improve cooperative decision-making and safety of connected autonomous vehicles at unsignalized intersections.

Contribution

It proposes a novel decentralized MARL framework with attention mechanisms and game priors, enhancing safety and efficiency in autonomous vehicle decision-making at intersections.

Findings

Outperforms baseline methods in safety and efficiency in simulations

Demonstrates improved safety and comfort in hardware-in-the-loop tests

Effectively models interaction dependencies among autonomous vehicles

Abstract

The development of autonomous vehicles has shown great potential to enhance the efficiency and safety of transportation systems. However, the decision-making issue in complex human-machine mixed traffic scenarios, such as unsignalized intersections, remains a challenge for autonomous vehicles. While reinforcement learning (RL) has been used to solve complex decision-making problems, existing RL methods still have limitations in dealing with cooperative decision-making of multiple connected autonomous vehicles (CAVs), ensuring safety during exploration, and simulating realistic human driver behaviors. In this paper, a novel and efficient algorithm, Multi-Agent Game-prior Attention Deep Deterministic Policy Gradient (MA-GA-DDPG), is proposed to address these limitations. Our proposed algorithm formulates the decision-making problem of CAVs at unsignalized intersections as a decentralized multi-agent reinforcement learning problem and incorporates an attention mechanism to capture interaction dependencies between ego CAV and other agents. The attention weights between the ego vehicle and other agents are then used to screen interaction objects and obtain prior hierarchical game relations, based on which a safety inspector module is designed to improve the traffic safety. Furthermore, both simulation and hardware-in-the-loop experiments were conducted, demonstrating that our method outperforms other baseline approaches in terms of driving safety, efficiency, and comfort.