Intramorphic Testing: A New Approach to the Test Oracle Problem

TL;DR

Intramorphic Testing is a novel white-box testing approach that creates modified system versions to generate test oracles, addressing the test oracle problem more effectively than existing black-box methods.

Contribution

This paper introduces Intramorphic Testing, a new white-box methodology that constructs modified systems to generate test oracles based on internal system changes.

Findings

Demonstrated Intramorphic Testing with use cases

Showed effectiveness in identifying faults

Compared favorably to black-box approaches

Abstract

A test oracle determines whether a system behaves correctly for a given input. Automatic testing techniques rely on an automated test oracle to test the system without user interaction. Important families of automated test oracles include Differential Testing and Metamorphic Testing, which are both black-box approaches; that is, they provide a test oracle that is oblivious to the system's internals. In this work, we propose Intramorphic Testing as a white-box methodology to tackle the test oracle problem. To realize an Intramorphic Testing approach, a modified version of the system is created, for which, given a single input, a test oracle can be provided that relates the output of the original and modified systems. As a concrete example, by replacing a greater-equals operator in the implementation of a sorting algorithm with smaller-equals, it would be expected that the output of the modified implementation is the reverse output of the original implementation. In this paper, we introduce the methodology and illustrate it via a set of use cases.