Mutation Testing Optimisations using the Clang Front-end

TL;DR

This paper explores optimization strategies using the Clang front-end to significantly reduce the time and computational costs of mutation testing for C language projects, making it more feasible.

Contribution

It introduces a set of optimization techniques based on Clang that eliminate compilation and execution overhead in mutation testing for C programs.

Findings

Achieved a speedup of 2x to 30x with the proposed optimizations.

Demonstrated feasibility of efficient mutation testing for C using Clang-based strategies.

Provided insights and lessons learned for future improvements.

Abstract

Mutation testing is the state-of-the-art technique for assessing the fault detection capacity of a test suite. Unfortunately, a full mutation analysis is often prohibitively expensive. The CppCheck project for instance, demands a build time of 5.8 minutes and a test execution time of 17 seconds on our desktop computer. An unoptimised mutation analysis, for 55,000 generated mutants took 11.8 days in total, of which 4.3 days is spent on (re)compiling the project. In this paper we present a feasibility study, investigating how a number of optimisation strategies can be implemented based on the Clang front-end. These optimisation strategies allow to eliminate the compilation and execution overhead in order to support efficient mutation testing for the C language family. We provide a proof-of-concept tool that achieves a speedup of between 2x and 30x. We make a detailed analysis of the speedup induced by the optimisations, elaborate on the lessons learned and point out avenues for further improvements.