SafePlanner: Testing Safety of the Automated Driving System Plan Model

TL;DR

SafePlanner is a systematic testing framework that identifies safety-critical flaws in automated driving system plans by generating meaningful scenarios and detecting hazardous behaviors, significantly improving coverage and bug detection.

Contribution

It introduces a structural analysis-based approach for scenario generation and hazard detection in ADS plan models, demonstrating effectiveness on Baidu Apollo.

Findings

Generated 20,635 test cases and found 520 hazardous behaviors.

Achieved 83.63% function coverage and 63.22% decision coverage.

Patches for four root causes eliminated issues without side effects.

Abstract

In this work, we present SafePlanner, a systematic testing framework for identifying safety-critical flaws in the Plan model of Automated Driving Systems (ADS). SafePlanner targets two core challenges: generating structurally meaningful test scenarios and detecting hazardous planning behaviors. To maximize coverage, SafePlanner performs a structural analysis of the Plan model implementation - specifically, its scene-transition logic and hierarchical control flow - and uses this insight to extract feasible scene transitions from code. It then composes test scenarios by combining these transitions with non-player vehicle (NPC) behaviors. Guided fuzzing is applied to explore the behavioral space of the Plan model under these scenarios. We evaluate SafePlanner on Baidu Apollo, a production-grade level 4 ADS. It generates 20635 test cases and detects 520 hazardous behaviors, grouped into 15 root causes through manual analysis. For four of these, we applied patches based on our analysis; the issues disappeared, and no apparent side effects were observed. SafePlanner achieves 83.63 percent function and 63.22 percent decision coverage on the Plan model, outperforming baselines in both bug discovery and efficiency.