Metamorphic Testing in Autonomous System Simulations

TL;DR

This paper explores the application of metamorphic testing to autonomous drone systems, demonstrating its effectiveness in identifying faults and improving quality assurance in simulation environments.

Contribution

It provides an overview of metamorphic testing, implements it in autonomous drone simulations, and identifies key properties and weaknesses in the system.

Findings

Metamorphic relations reveal implementation weaknesses.

Testing improves understanding of obstacle avoidance algorithms.

Potential for enhanced quality assurance in autonomous systems.

Abstract

Metamorphic testing has proven to be effective for test case generation and fault detection in many domains. It is a software testing strategy that uses certain relations between input-output pairs of a program, referred to as metamorphic relations. This approach is relevant in the autonomous systems domain since it helps in cases where the outcome of a given test input may be difficult to determine. In this paper therefore, we provide an overview of metamorphic testing as well as an implementation in the autonomous systems domain. We implement an obstacle detection and avoidance task in autonomous drones utilising the GNC API alongside a simulation in Gazebo. Particularly, we describe properties and best practices that are crucial for the development of effective metamorphic relations. We also demonstrate two metamorphic relations for metamorphic testing of single and more than one drones, respectively. Our relations reveal several properties and some weak spots of both the implementation and the avoidance algorithm in the light of metamorphic testing. The results indicate that metamorphic testing has great potential in the autonomous systems domain and should be considered for quality assurance in this field.