A Differentiated Reward Method for Reinforcement Learning based Multi-Vehicle Cooperative Decision-Making Algorithms

TL;DR

This paper introduces a differentiated reward method for reinforcement learning that improves training efficiency and decision-making in multi-vehicle cooperative driving, demonstrating enhanced traffic safety and efficiency.

Contribution

It proposes a novel reward design based on steady-state transition systems that incorporates gradient information, improving RL performance in multi-vehicle scenarios.

Findings

Accelerates training convergence in RL algorithms.

Outperforms existing reward methods in traffic efficiency and safety.

Shows strong scalability and adaptability in complex traffic environments.

Abstract

Reinforcement learning (RL) shows great potential for optimizing multi-vehicle cooperative driving strategies through the state-action-reward feedback loop, but it still faces challenges such as low sample efficiency. This paper proposes a differentiated reward method based on steady-state transition systems, which incorporates state transition gradient information into the reward design by analyzing traffic flow characteristics, aiming to optimize action selection and policy learning in multi-vehicle cooperative decision-making. The performance of the proposed method is validated in RL algorithms such as MAPPO, MADQN, and QMIX under varying autonomous vehicle penetration. The results show that the differentiated reward method significantly accelerates training convergence and outperforms centering reward and others in terms of traffic efficiency, safety, and action rationality. Additionally, the method demonstrates strong scalability and environmental adaptability, providing a novel approach for multi-agent cooperative decision-making in complex traffic scenarios.