Reinforcement Learning with Iterative Reasoning for Merging in Dense Traffic

TL;DR

This paper introduces a novel approach combining reinforcement learning and game theory to improve autonomous vehicle merging in dense traffic by training with level-$k$ behavior to enhance robustness and efficiency.

Contribution

It proposes a new training curriculum using level-$k$ reasoning in reinforcement learning for autonomous merging in dense traffic, improving policy robustness.

Findings

Learned policies are more efficient than traditional methods.

Training with level-$k$ behavior enhances robustness to diverse traffic scenarios.

Approach outperforms baseline methods in simulation.

Abstract

Maneuvering in dense traffic is a challenging task for autonomous vehicles because it requires reasoning about the stochastic behaviors of many other participants. In addition, the agent must achieve the maneuver within a limited time and distance. In this work, we propose a combination of reinforcement learning and game theory to learn merging behaviors. We design a training curriculum for a reinforcement learning agent using the concept of level-$k$ behavior. This approach exposes the agent to a broad variety of behaviors during training, which promotes learning policies that are robust to model discrepancies. We show that our approach learns more efficient policies than traditional training methods.