Evaluating Random Mutant Selection at Class-Level in Projects with Non-Adequate Test Suites

TL;DR

This paper investigates the effectiveness of random mutant selection at class level in projects with non-adequate test suites, proposing a weighted approach to improve representativeness and analyzing its impact on different test adequacy levels.

Contribution

It introduces a weighted random mutant selection method and empirically evaluates its performance on projects with varying test suite adequacy levels.

Findings

Uniform random mutant selection underperforms on non-adequate test suites.

Weighted random mutant selection produces more representative mutant samples.

Projects with higher test adequacy yield larger representative mutant samples.

Abstract

Mutation testing is a standard technique to evaluate the quality of a test suite. Due to its computationally intensive nature, many approaches have been proposed to make this technique feasible in real case scenarios. Among these approaches, uniform random mutant selection has been demonstrated to be simple and promising. However, works on this area analyze mutant samples at project level mainly on projects with adequate test suites. In this paper, we fill this lack of empirical validation by analyzing random mutant selection at class level on projects with non-adequate test suites. First, we show that uniform random mutant selection underachieves the expected results. Then, we propose a new approach named weighted random mutant selection which generates more representative mutant samples. Finally, we show that representative mutant samples are larger for projects with high test adequacy.