OpenSBT: A Modular Framework for Search-based Testing of Automated Driving Systems

TL;DR

OpenSBT is a modular, open-source framework that simplifies search-based testing of automated driving systems by integrating simulators, search algorithms, and fitness functions, demonstrated on emergency braking systems.

Contribution

It introduces a flexible, extensible framework for ADS testing that overcomes integration challenges and enables empirical comparisons across simulators and search methods.

Findings

Successfully tested emergency braking in CARLA and Prescan

Facilitated empirical comparisons of search algorithms

Enhanced accessibility for ADS testing research

Abstract

Search-based software testing (SBT) is an effective and efficient approach for testing automated driving systems (ADS). However, testing pipelines for ADS testing are particularly challenging as they involve integrating complex driving simulation platforms and establishing communication protocols and APIs with the desired search algorithm. This complexity prevents a wide adoption of SBT and thorough empirical comparative experiments with different simulators and search approaches. We present OpenSBT, an open-source, modular and extensible framework to facilitate the SBT of ADS. With OpenSBT, it is possible to integrate simulators with an embedded system under test, search algorithms and fitness functions for testing. We describe the architecture and show the usage of our framework by applying different search algorithms for testing Automated Emergency Braking Systems in CARLA as well in the high-fidelity Prescan simulator in collaboration with our industrial partner DENSO. OpenSBT is available at https://git.fortiss.org/opensbt. A demo video is provided here: https://youtu.be/6csl\_UAOD\_4.