Latent Mutants: A large-scale study on the Interplay between mutation testing and software evolution

TL;DR

This study investigates how mutants in software evolve over time, focusing on latent mutants that are initially live but later killed, and demonstrates that they can be predicted with high accuracy using machine learning.

Contribution

It introduces the concept of latent mutants in mutation testing and shows they can be effectively predicted using program change features and machine learning.

Findings

11.2% of mutants are live in current version

3.5% of mutants are latent, appearing after 104 days on average

Latent mutants can be predicted with 86% accuracy

Abstract

In this paper we apply mutation testing in an in-time fashion, i.e., across multiple project releases. Thus, we investigate how the mutants of the current version behave in the future versions of the programs. We study the characteristics of what we call latent mutants, i.e., the mutants that are live in one version and killed in later revisions, and explore whether they are predictable with these properties. We examine 131,308 mutants generated by Pitest on 13 open-source projects. Around 11.2% of these mutants are live, and 3.5% of them are latent, manifesting in 104 days on average. Using the mutation operators and change-related features we successfully demonstrate that these latent mutants are identifiable, predicting them with an accuracy of 86% and a balanced accuracy of 67% using a simple random forest classifier.