Towards Automatic Generation of Amplified Regression Test Oracles

TL;DR

This paper presents a novel approach to automatically generate amplified regression test oracles by leveraging object state data during test execution, improving the detection of software behavior changes.

Contribution

It introduces a test oracle derivation method that uses object state monitoring to enhance regression testing without relying on predefined oracles.

Findings

Enhanced detection of behavior changes in preliminary evaluation

Uses object state data to improve test oracle quality

Potential to reduce testing costs and improve reliability

Abstract

Regression testing is crucial in ensuring that pure code refactoring does not adversely affect existing software functionality, but it can be expensive, accounting for half the cost of software maintenance. Automated test case generation reduces effort but may generate weak test suites. Test amplification is a promising solution that enhances tests by generating additional or improving existing ones, increasing test coverage, but it faces the test oracle problem. To address this, we propose a test oracle derivation approach that uses object state data produced during System Under Test (SUT) test execution to amplify regression test oracles. The approach monitors the object state during test execution and compares it to the previous version to detect any changes in relation to the SUT's intended behaviour. Our preliminary evaluation shows that the proposed approach can enhance the detection of behaviour changes substantially, providing initial evidence of its effectiveness.