A Two-stage Based Social Preference Recognition in Multi-Agent Autonomous Driving System

TL;DR

This paper introduces a two-stage framework for multi-agent autonomous driving that enables agents to recognize each other's social preferences, leading to improved coordination and traffic flow in complex scenarios.

Contribution

It proposes a novel two-stage system that trains agents with shared SVOs and then recognizes other agents' SVOs to enhance multi-agent cooperation.

Findings

Significant performance improvement over state-of-the-art MARL algorithms.

Effective SVO recognition enhances traffic coordination.

Framework applicable to dense multi-agent driving scenarios.

Abstract

Multi-Agent Reinforcement Learning (MARL) has become a promising solution for constructing a multi-agent autonomous driving system (MADS) in complex and dense scenarios. But most methods consider agents acting selfishly, which leads to conflict behaviors. Some existing works incorporate the concept of social value orientation (SVO) to promote coordination, but they lack the knowledge of other agents' SVOs, resulting in conservative maneuvers. In this paper, we aim to tackle the mentioned problem by enabling the agents to understand other agents' SVOs. To accomplish this, we propose a two-stage system framework. Firstly, we train a policy by allowing the agents to share their ground truth SVOs to establish a coordinated traffic flow. Secondly, we develop a recognition network that estimates agents' SVOs and integrates it with the policy trained in the first stage. Experiments demonstrate that our developed method significantly improves the performance of the driving policy in MADS compared to two state-of-the-art MARL algorithms.