Exploring a Test Data-Driven Method for Selecting and Constraining Metamorphic Relations

TL;DR

This paper introduces MetaTrimmer, a test data-driven approach for selecting and constraining metamorphic relations in testing, which avoids complex models and integrates well with fuzzing to improve effectiveness.

Contribution

MetaTrimmer is a novel method that uses test data transformations and manual inspection to select and constrain MRs without relying on labeled datasets or complex prediction models.

Findings

MetaTrimmer can identify effective MRs from test data.

It avoids the need for labeled datasets or complex models.

Preliminary results show potential to improve MR selection.

Abstract

Identifying and selecting high-quality Metamorphic Relations (MRs) is a challenge in Metamorphic Testing (MT). While some techniques for automatically selecting MRs have been proposed, they are either domain-specific or rely on strict assumptions about the applicability of a pre-defined MRs. This paper presents a preliminary evaluation of MetaTrimmer, a method for selecting and constraining MRs based on test data. MetaTrimmer comprises three steps: generating random test data inputs for the SUT (Step 1), performing test data transformations and logging MR violations (Step 2), and conducting manual inspections to derive constraints (Step 3). The novelty of MetaTrimmer is its avoidance of complex prediction models that require labeled datasets regarding the applicability of MRs. Moreover, MetaTrimmer facilitates the seamless integration of MT with advanced fuzzing for test data generation. In a preliminary evaluation, MetaTrimmer shows the potential to overcome existing limitations and enhance MR effectiveness.