Concepts in Testing of Autonomous Systems: Academic Literature and Industry Practice

TL;DR

This paper explores the testing of autonomous systems by synthesizing academic literature and industry insights, identifying challenges, current practices, and highlighting the need for further research to enhance safety and quality.

Contribution

It provides a comprehensive conceptual framework, classifications, and analysis of testing practices and challenges in autonomous systems, bridging academia and industry insights.

Findings

Identification of key challenges in testing autonomous systems

Classification of current testing practices and techniques

Highlighting the need for further research to improve safety and quality

Abstract

Testing of autonomous systems is extremely important as many of them are both safety-critical and security-critical. The architecture and mechanism of such systems are fundamentally different from traditional control software, which appears to operate in more structured environments and are explicitly instructed according to the system design and implementation. To gain a better understanding of autonomous systems practice and facilitate research on testing of such systems, we conducted an exploratory study by synthesizing academic literature with a focus group discussion and interviews with industry practitioners. Based on thematic analysis of the data, we provide a conceptualization of autonomous systems, classifications of challenges and current practices as well as of available techniques and approaches for testing of autonomous systems. Our findings also indicate that more research efforts are required for testing of autonomous systems to improve both the quality and safety aspects of such systems.