A Roadmap for Simulation-Based Testing of Autonomous Cyber-Physical Systems: Challenges and Future Direction

TL;DR

This paper presents a strategic roadmap for simulation-based testing of autonomous cyber-physical systems, addressing unique challenges and proposing future research directions to improve safety and reliability.

Contribution

It introduces a comprehensive framework for simulation testing of ACPSs, emphasizing test automation, benchmark creation, and tailored tools for autonomous systems.

Findings

Identifies key challenges in simulation-based testing of ACPSs

Proposes concrete definitions for test cases in simulation environments

Highlights the need for new benchmarks and automated tools

Abstract

As the era of autonomous cyber-physical systems (ACPSs), such as unmanned aerial vehicles and self-driving cars, unfolds, the demand for robust testing methodologies is key to realizing the adoption of such systems in real-world scenarios. However, traditional software testing paradigms face unprecedented challenges in ensuring the safety and reliability of these systems. In response, this paper pioneers a strategic roadmap for simulation-based testing of ACPSs, specifically focusing on autonomous systems. Our paper discusses the relevant challenges and obstacles of ACPSs, focusing on test automation and quality assurance, hence advocating for tailored solutions to address the unique demands of autonomous systems. While providing concrete definitions of test cases within simulation environments, we also accentuate the need to create new benchmark assets and the development of automated tools tailored explicitly for autonomous systems in the software engineering community. This paper not only highlights the relevant, pressing issues the software engineering community should focus on (in terms of practices, expected automation, and paradigms), but it also outlines ways to tackle them. By outlining the various domains and challenges of simulation-based testing/development for ACPSs, we provide directions for future research efforts.