Analyzing and Enhancing Closed-loop Stability in Reactive Simulation

TL;DR

This paper analyzes the stability issues in reactive simulation for self-driving vehicles and proposes a new framework with kinematic models and novel metrics to enhance stability and better evaluate simulation performance.

Contribution

The paper introduces a reactive simulation framework with stability analysis and incorporates kinematic vehicle models to improve closed-loop stability.

Findings

Stability is linked to smoothness and consistency of state sequences.

Incorporating kinematic models improves simulation stability.

New metrics provide better analysis of simulation performance.

Abstract

Simulation has played an important role in efficiently evaluating self-driving vehicles in terms of scalability. Existing methods mostly rely on heuristic-based simulation, where traffic participants follow certain human-encoded rules that fail to generate complex human behaviors. Therefore, the reactive simulation concept is proposed to bridge the human behavior gap between simulation and real-world traffic scenarios by leveraging real-world data. However, these reactive models can easily generate unreasonable behaviors after a few steps of simulation, where we regard the model as losing its stability. To the best of our knowledge, no work has explicitly discussed and analyzed the stability of the reactive simulation framework. In this paper, we aim to provide a thorough stability analysis of the reactive simulation and propose a solution to enhance the stability. Specifically, we first propose a new reactive simulation framework, where we discover that the smoothness and consistency of the simulated state sequences are crucial factors to stability. We then incorporate the kinematic vehicle model into the framework to improve the closed-loop stability of the reactive simulation. Furthermore, along with commonly-used metrics, several novel metrics are proposed in this paper to better analyze the simulation performance.