Detection of Coincidentally Correct Test Cases through Random Forests

TL;DR

This paper introduces a hybrid ensemble learning method using Random Forests to identify and mitigate the impact of coincidentally correct test cases, improving the accuracy of fault localization.

Contribution

It proposes a novel approach combining ensemble learning and supervised classification to detect and handle coincidentally correct test cases in fault localization.

Findings

Effective identification of coincidentally correct test cases.

Improved fault localization accuracy after handling these test cases.

Cost analysis of different mitigation strategies.

Abstract

The performance of coverage-based fault localization greatly depends on the quality of test cases being executed. These test cases execute some lines of the given program and determine whether the underlying tests are passed or failed. In particular, some test cases may be well-behaved (i.e., passed) while executing faulty statements. These test cases, also known as coincidentally correct test cases, may negatively influence the performance of the spectra-based fault localization and thus be less helpful as a tool for the purpose of automated debugging. In other words, the involvement of these coincidentally correct test cases may introduce noises to the fault localization computation and thus cause in divergence of effectively localizing the location of possible bugs in the given code. In this paper, we propose a hybrid approach of ensemble learning combined with a supervised learning algorithm namely, Random Forests (RF) for the purpose of correctly identifying test cases that are mislabeled to be the passing test cases. A cost-effective analysis of flipping the test status or trimming (i.e., eliminating from the computation) the coincidental correct test cases is also reported.