Objective-aware Traffic Simulation via Inverse Reinforcement Learning

TL;DR

This paper introduces a novel inverse reinforcement learning approach for traffic simulation that adapts to different traffic dynamics and accurately imitates real vehicle behaviors, surpassing traditional fixed-model simulators.

Contribution

It proposes a parameter sharing adversarial inverse reinforcement learning model that learns invariant reward functions for robust traffic simulation across diverse environments.

Findings

Outperforms state-of-the-art methods in synthetic and real-world datasets.

Demonstrates robustness to varying traffic dynamics.

Recovers vehicle objectives invariant to environment changes.

Abstract

Traffic simulators act as an essential component in the operating and planning of transportation systems. Conventional traffic simulators usually employ a calibrated physical car-following model to describe vehicles' behaviors and their interactions with traffic environment. However, there is no universal physical model that can accurately predict the pattern of vehicle's behaviors in different situations. A fixed physical model tends to be less effective in a complicated environment given the non-stationary nature of traffic dynamics. In this paper, we formulate traffic simulation as an inverse reinforcement learning problem, and propose a parameter sharing adversarial inverse reinforcement learning model for dynamics-robust simulation learning. Our proposed model is able to imitate a vehicle's trajectories in the real world while simultaneously recovering the reward function that reveals the vehicle's true objective which is invariant to different dynamics. Extensive experiments on synthetic and real-world datasets show the superior performance of our approach compared to state-of-the-art methods and its robustness to variant dynamics of traffic.