Considering Polymorphism in Change-Based Test Suite Reduction

TL;DR

This paper explores how incorporating polymorphism into change-based test suite reduction algorithms improves test relevance but also increases the size of the test suite, affecting execution time.

Contribution

It introduces a polymorphism-aware approach to test suite reduction, addressing false negatives in previous change-based algorithms.

Findings

Polymorphism improves test relevance and accuracy.

Increased test suite size when using polymorphism.

Trade-off between test suite relevance and size.

Abstract

With the increasing popularity of continuous integration, algorithms for selecting the minimal test-suite to cover a given set of changes are in order. This paper reports on how polymorphism can handle false negatives in a previous algorithm which uses method-level changes in the base-code to deduce which tests need to be rerun. We compare the approach with and without polymorphism on two distinct cases ---PMD and CruiseControl--- and discovered an interesting trade-off: incorporating polymorphism results in more relevant tests to be included in the test suite (hence improves accuracy), however comes at the cost of a larger test suite (hence increases the time to run the minimal test-suite).