A Review of Testing Object-Based Environment Perception for Safe Automated Driving

TL;DR

This paper reviews existing methods for testing environment perception in automated driving, highlighting the challenges and open issues in ensuring safety through perception verification and validation.

Contribution

It provides a structured overview of perception testing approaches, standards, and benchmarks, identifying key open challenges in safety-aware perception testing.

Findings

Safety-aware perception testing remains an open issue.

Challenges exist across test criteria, scenarios, and reference data.

Interdependencies between testing axes are not yet fully addressed.

Abstract

Safety assurance of automated driving systems must consider uncertain environment perception. This paper reviews literature addressing how perception testing is realized as part of safety assurance. We focus on testing for verification and validation purposes at the interface between perception and planning, and structure our analysis along the three axes 1) test criteria and metrics, 2) test scenarios, and 3) reference data. Furthermore, the analyzed literature includes related safety standards, safety-independent perception algorithm benchmarking, and sensor modeling. We find that the realization of safety-aware perception testing remains an open issue since challenges concerning the three testing axes and their interdependencies currently do not appear to be sufficiently solved.