DynNPC: Finding More Violations Induced by ADS in Simulation Testing through Dynamic NPC Behavior Generation

TL;DR

DynNPC is a dynamic scenario testing framework for autonomous driving systems that generates more realistic NPC behaviors during simulation, leading to the discovery of more violations and improving testing effectiveness.

Contribution

The paper introduces DynNPC, a novel approach that dynamically generates NPC behaviors during simulation based on traffic signals and Ego vehicle actions, unlike prior static methods.

Findings

DynNPC finds more violation scenarios than existing methods.

DynNPC improves testing efficiency in autonomous driving simulation.

DynNPC outperforms five state-of-the-art approaches.

Abstract

Recently, a number of simulation testing approaches have been proposed to generate diverse driving scenarios for autonomous driving systems (ADSs) testing. However, the behaviors of NPC vehicles in these scenarios generated by previous approaches are predefined and mutated before simulation execution, ignoring traffic signals and the behaviors of the Ego vehicle. Thus, a large number of the violations they found are induced by unrealistic behaviors of NPC vehicles, revealing no bugs of ADSs. Besides, the vast scenario search space of NPC behaviors during the iterative mutations limits the efficiency of previous approaches. To address these limitations, we propose a novel scenario-based testing framework, DynNPC, to generate more violation scenarios induced by the ADS. Specifically, DynNPC allows NPC vehicles to dynamically generate behaviors using different driving strategies during simulation execution based on traffic signals and the real-time behavior of the Ego vehicle. We compare DynNPC with five state-of-the-art scenario-based testing approaches. Our evaluation has demonstrated the effectiveness and efficiency of DynNPC in finding more violation scenarios induced by the ADS.