SOTIF-Compliant Scenario Generation Using Semi-Concrete Scenarios and Parameter Sampling

TL;DR

This paper introduces a novel approach combining semi-concrete scenarios with parameter sampling to generate more effective and comprehensive test suites for verifying autonomous driving systems, aligning with SOTIF standards.

Contribution

It proposes semi-concrete scenarios and parameter sampling to balance exploration and exploitation in scenario generation for SOTIF compliance.

Findings

Generated more effective test suites than existing methods.

Including feedback in parameter sampling improves test suite quality.

Achieved higher coverage and fault detection in experiments.

Abstract

The SOTIF standard (ISO 21448) requires scenario-based testing to verify and validate Advanced Driver Assistance Systems and Automated Driving Systems but does not suggest any practical way to do so effectively and efficiently. Existing scenario generation approaches either focus on exploring or exploiting the scenario space. This generally leads to test suites that cover many known cases but potentially miss edge cases or focused test suites that are effective but also contain less diverse scenarios. To generate SOTIF-compliant test suites that achieve higher coverage and find more faults, this paper proposes semi-concrete scenarios and combines them with parameter sampling to adequately balance scenario space exploration and exploitation. Semi-concrete scenarios enable combinatorial scenario generation techniques that systematically explore the scenario space, while parameter sampling allows for the exploitation of continuous parameters. Our experimental results show that the proposed concept can generate more effective test suites than state-of-the-art coverage-based sampling. Moreover, our results show that including a feedback mechanism to drive parameter sampling further increases test suites' effectiveness.