AI-Augmented Metamorphic Testing for Comprehensive Validation of Autonomous Vehicles

TL;DR

This paper proposes an AI-augmented metamorphic testing approach for autonomous vehicles, using AI-driven image generation to create diverse, realistic testing scenarios that improve safety validation beyond traditional methods.

Contribution

It introduces integrating AI-generated scenario variations into metamorphic testing to enhance the thoroughness and realism of autonomous vehicle validation.

Findings

Enables reproducible and diverse scenario testing

Improves coverage of real-world conditions

Addresses limitations of traditional testing methods

Abstract

Self-driving cars have the potential to revolutionize transportation, but ensuring their safety remains a significant challenge. These systems must navigate a variety of unexpected scenarios on the road, and their complexity poses substantial difficulties for thorough testing. Conventional testing methodologies face critical limitations, including the oracle problem determining whether the systems behavior is correct and the inability to exhaustively recreate a range of situations a self-driving car may encounter. While Metamorphic Testing (MT) offers a partial solution to these challenges, its application is often limited by simplistic modifications to test scenarios. In this position paper, we propose enhancing MT by integrating AI-driven image generation tools, such as Stable Diffusion, to improve testing methodologies. These tools can generate nuanced variations of driving scenarios within the operational design domain (ODD)for example, altering weather conditions, modifying environmental elements, or adjusting lane markings while preserving the critical features necessary for system evaluation. This approach enables reproducible testing, efficient reuse of test criteria, and comprehensive evaluation of a self-driving systems performance across diverse scenarios, thereby addressing key gaps in current testing practices.