Reinforcement Learning based Negotiation-aware Motion Planning of Autonomous Vehicles

TL;DR

This paper introduces a reinforcement learning framework for autonomous vehicle motion planning that adaptively negotiates with human traffic participants by understanding intentions and adjusting driving styles in real time.

Contribution

It presents a novel RL-based negotiation-aware motion planning method that dynamically modifies prediction horizons and models interactions as a Markov Decision Process.

Findings

Outperforms traditional methods in simulation and real-world tests.

Effectively alleviates social dilemma problems in traffic scenarios.

Enhances robustness through curriculum learning.

Abstract

For autonomous vehicles integrating onto roadways with human traffic participants, it requires understanding and adapting to the participants' intention and driving styles by responding in predictable ways without explicit communication. This paper proposes a reinforcement learning based negotiation-aware motion planning framework, which adopts RL to adjust the driving style of the planner by dynamically modifying the prediction horizon length of the motion planner in real time adaptively w.r.t the event of a change in environment, typically triggered by traffic participants' switch of intents with different driving styles. The framework models the interaction between the autonomous vehicle and other traffic participants as a Markov Decision Process. A temporal sequence of occupancy grid maps are taken as inputs for RL module to embed an implicit intention reasoning. Curriculum learning is employed to enhance the training efficiency and the robustness of the algorithm. We applied our method to narrow lane navigation in both simulation and real world to demonstrate that the proposed method outperforms the common alternative due to its advantage in alleviating the social dilemma problem with proper negotiation skills.