Learning Reward Models for Cooperative Trajectory Planning with Inverse Reinforcement Learning and Monte Carlo Tree Search

TL;DR

This paper introduces a method combining inverse reinforcement learning and Monte Carlo Tree Search to learn reward models for cooperative trajectory planning, enabling automated vehicles to behave more human-like in traffic scenarios.

Contribution

It presents a novel approach that integrates feature-based maximum entropy inverse reinforcement learning with Monte Carlo Tree Search for reward modeling in cooperative planning.

Findings

Successfully recovers reward models that mimic expert trajectories.

Performs comparably to manually tuned baseline reward models.

Enhances predictability of automated vehicle behavior in traffic.

Abstract

Cooperative trajectory planning methods for automated vehicles can solve traffic scenarios that require a high degree of cooperation between traffic participants. However, for cooperative systems to integrate into human-centered traffic, the automated systems must behave human-like so that humans can anticipate the system's decisions. While Reinforcement Learning has made remarkable progress in solving the decision-making part, it is non-trivial to parameterize a reward model that yields predictable actions. This work employs feature-based Maximum Entropy Inverse Reinforcement Learning combined with Monte Carlo Tree Search to learn reward models that maximize the likelihood of recorded multi-agent cooperative expert trajectories. The evaluation demonstrates that the approach can recover a reasonable reward model that mimics the expert and performs similarly to a manually tuned baseline reward model.