Partial Attention in Deep Reinforcement Learning for Safe Multi-Agent Control

TL;DR

This paper introduces a partial attention mechanism within a QMIX framework for multi-agent deep reinforcement learning to improve safety and efficiency in autonomous vehicle highway merging scenarios.

Contribution

It presents a novel neural network design incorporating partial attention for multi-agent control in Dec-POMDP environments, enhancing safety and performance.

Findings

Improved safety metrics in simulations

Higher driving speeds achieved

Enhanced reward optimization

Abstract

Attention mechanisms excel at learning sequential patterns by discriminating data based on relevance and importance. This provides state-of-the-art performance in advanced generative artificial intelligence models. This paper applies this concept of an attention mechanism for multi-agent safe control. We specifically consider the design of a neural network to control autonomous vehicles in a highway merging scenario. The environment is modeled as a Decentralized Partially Observable Markov Decision Process (Dec-POMDP). Within a QMIX framework, we include partial attention for each autonomous vehicle, thus allowing each ego vehicle to focus on the most relevant neighboring vehicles. Moreover, we propose a comprehensive reward signal that considers the global objectives of the environment (e.g., safety and vehicle flow) and the individual interests of each agent. Simulations are conducted in the Simulation of Urban Mobility (SUMO). The results show better performance compared to other driving algorithms in terms of safety, driving speed, and reward.