Analyzing Closed-loop Training Techniques for Realistic Traffic Agent Models in Autonomous Highway Driving Simulations

TL;DR

This paper compares various closed-loop training techniques for creating realistic traffic agent models in autonomous highway driving simulations, aiming to improve simulation fidelity and safety.

Contribution

It provides an extensive comparative analysis of different training principles, including open-loop, closed-loop, adversarial, and reinforcement methods, for traffic agent modeling.

Findings

Closed-loop training improves realism over open-loop methods.

Certain combinations of training techniques yield more accurate traffic behaviors.

Reinforcement losses and adversarial training enhance agent performance.

Abstract

Simulation plays a crucial role in the rapid development and safe deployment of autonomous vehicles. Realistic traffic agent models are indispensable for bridging the gap between simulation and the real world. Many existing approaches for imitating human behavior are based on learning from demonstration. However, these approaches are often constrained by focusing on individual training strategies. Therefore, to foster a broader understanding of realistic traffic agent modeling, in this paper, we provide an extensive comparative analysis of different training principles, with a focus on closed-loop methods for highway driving simulation. We experimentally compare (i) open-loop vs. closed-loop multi-agent training, (ii) adversarial vs. deterministic supervised training, (iii) the impact of reinforcement losses, and (iv) the impact of training alongside log-replayed agents to identify suitable training techniques for realistic agent modeling. Furthermore, we identify promising combinations of different closed-loop training methods.