On the Verification and Validation of AI Navigation Algorithms

TL;DR

This paper reviews current methods for verifying and validating autonomous surface ship navigation algorithms, highlighting the reliance on limited simulations and proposing a systematic scenario-based testing approach for more comprehensive validation.

Contribution

It provides a systematic mapping of recent research on navigation algorithms and introduces a new scenario-based testing method for improved validation.

Findings

Most research uses simulations for validation

Simulations often involve few manually designed scenarios

Proposes systematic scenario-based testing for better validation

Abstract

This paper explores the state of the art on to methods to verify and validate navigation algorithms for autonomous surface ships. We perform a systematic mapping study to find research works published in the last 10 years proposing new algorithms for autonomous navigation and collision avoidance and we have extracted what verification and validation approaches have been applied on these algorithms. We observe that most research works use simulations to validate their algorithms. However, these simulations often involve just a few scenarios designed manually. This raises the question if the algorithms have been validated properly. To remedy this, we propose the use of a systematic scenario-based testing approach to validate navigation algorithms extensively.