Reformulating Regression Test Suite Optimization using Quantum Annealing -- an Empirical Study

TL;DR

This paper introduces SelectQA, a quantum annealing method for regression test suite optimization that aims to improve effectiveness and efficiency over traditional and existing quantum approaches, demonstrated through empirical evaluation.

Contribution

It reformulates the regression test case selection problem to leverage quantum annealing, achieving better effectiveness than previous quantum methods and comparable efficiency to classical algorithms.

Findings

SelectQA outperforms BootQA in effectiveness.

SelectQA is more efficient than DIV-GA.

SelectQA has similar efficiency to Additional Greedy.

Abstract

Maintaining software quality is crucial in the dynamic landscape of software development. Regression testing ensures that software works as expected after changes are implemented. However, re-executing all test cases for every modification is often impractical and costly, particularly for large systems. Although very effective, traditional test suite optimization techniques are often impractical in resource-constrained scenarios, as they are computationally expensive. Hence, quantum computing solutions have been developed to improve their efficiency but have shown drawbacks in terms of effectiveness. We propose reformulating the regression test case selection problem to use quantum computation techniques better. Our objectives are (i) to provide more efficient solutions than traditional methods and (ii) to improve the effectiveness of previously proposed quantum-based solutions. We propose SelectQA, a quantum annealing approach that can outperform the quantum-based approach BootQA in terms of effectiveness while obtaining results comparable to those of the classic Additional Greedy and DIV-GA approaches. Regarding efficiency, SelectQA outperforms DIV-GA and has similar results with the Additional Greedy algorithm but is exceeded by BootQA.