Test case prioritization using test case diversification and fault-proneness estimations

TL;DR

This paper introduces a test case prioritization approach that combines test case diversification, coverage data, and fault-proneness estimations to improve regression testing efficiency on real-world projects.

Contribution

It proposes a novel TCP method integrating diversification and fault-proneness, evaluated on multiple open-source projects, outperforming traditional coverage-based methods.

Findings

Proposed methods outperform coverage-based TCP approaches.

Combining diversification with fault-proneness improves prioritization.

Evaluation on 357 versions from five projects demonstrates effectiveness.

Abstract

Regression testing activities greatly reduce the risk of faulty software release. However, the size of the test suites grows throughout the development process, resulting in time-consuming execution of the test suite and delayed feedback to the software development team. This has urged the need for approaches such as test case prioritization (TCP) and test-suite reduction to reach better results in case of limited resources. In this regard, proposing approaches that use auxiliary sources of data such as bug history can be interesting. We aim to propose an approach for TCP that takes into account test case coverage data, bug history, and test case diversification. To evaluate this approach we study its performance on real-world open-source projects. The bug history is used to estimate the fault-proneness of source code areas. The diversification of test cases is preserved by incorporating fault-proneness on a clustering-based approach scheme. The proposed methods are evaluated on datasets collected from the development history of five real-world projects including 357 versions in total. The experiments show that the proposed methods are superior to coverage-based TCP methods. The proposed approach shows that improvement of coverage-based and fault-proneness-based methods is possible by using a combination of diversification and fault-proneness incorporation.