AmbieGen: A Search-based Framework for Autonomous Systems Testing

TL;DR

AmbieGen is a modular, search-based framework that uses evolutionary algorithms to generate critical test scenarios for autonomous systems, improving safety testing efficiency.

Contribution

This paper introduces AmbieGen, a novel framework employing evolutionary search for automated test case generation in autonomous systems testing.

Findings

Successfully generates critical test scenarios for autonomous robots and cars.

Demonstrates practical application of the framework in safety testing.

Provides a flexible architecture for extending to other systems.

Abstract

Thorough testing of safety-critical autonomous systems, such as self-driving cars, autonomous robots, and drones, is essential for detecting potential failures before deployment. One crucial testing stage is model-in-the-loop testing, where the system model is evaluated by executing various scenarios in a simulator. However, the search space of possible parameters defining these test scenarios is vast, and simulating all combinations is computationally infeasible. To address this challenge, we introduce AmbieGen, a search-based test case generation framework for autonomous systems. AmbieGen uses evolutionary search to identify the most critical scenarios for a given system, and has a modular architecture that allows for the addition of new systems under test, algorithms, and search operators. Currently, AmbieGen supports test case generation for autonomous robots and autonomous car lane keeping assist systems. In this paper, we provide a high-level overview of the framework's architecture and demonstrate its practical use cases.