Towards Automated Metamorphic Test Identification for Ocean System Models

TL;DR

This paper proposes an automated machine learning-based approach to generate metamorphic tests for ocean system models, addressing the challenge of lacking test oracles and reducing computational complexity.

Contribution

It extends previous methods by incorporating dimensionality reduction and mutation techniques to efficiently identify diverse metamorphic relations in large ocean modeling datasets.

Findings

Successfully applied to ocean-modeling applications

Reduced computational complexity in metamorphic relation identification

Demonstrated effectiveness in testing multiple implementations

Abstract

Metamorphic testing seeks to verify software in the absence of test oracles. Our application domain is ocean system modeling, where test oracles rarely exist, but where symmetries of the simulated physical systems are known. The input data set is large owing to the requirements of the application domain. This paper presents work in progress for the automated generation of metamorphic test scenarios using machine learning. We extended our previously proposed method [1] to identify metamorphic relations with reduced computational complexity. Initially, we represent metamorphic relations as identity maps. We construct a cost function that minimizes for identifying a metamorphic relation orthogonal to previously found metamorphic relations and penalize for the identity map. A machine learning algorithm is used to identify all possible metamorphic relations minimizing the defined cost function. We propose applying dimensionality reduction techniques to identify attributes in the input which have high variance among the identified metamorphic relations. We apply mutation on these selected attributes to identify distinct metamorphic relations with reduced computational complexity. For experimental evaluation, we subject the two implementations of an ocean-modeling application to the proposed method to present the use of metamorphic relations to test the two implementations of this application.