Cloud-Based Distributed Mutation Analysis

TL;DR

This paper presents a cloud-based distributed mutation testing system that significantly reduces testing time by leveraging cloud clusters and MapReduce, outperforming existing non-distributed tools.

Contribution

It introduces a novel architecture and prototype for distributed mutation testing using cloud computing and evaluates its performance against existing tools.

Findings

Prototype outperforms PiT in mutation testing speed

Work distribution strategies impact performance efficiency

Potential for further performance improvements identified

Abstract

Mutation Testing is a fault-based software testing technique which is too computationally expensive for industrial use. Cloud-based distributed computing clusters, taking advantage of the MapReduce programming paradigm, represent a method by which the long running time can be reduced. In this paper, we describe an architecture, and a prototype implementation, of such a cloud-based distributed mutation testing system. To evaluate the system, we compared the performance of the prototype, with various cluster sizes, to an existing "state-of-the-art" non-distributed tool, PiT. We also analysed different approaches to work distribution, to determine how to most efficiently divide the mutation analysis task. Our tool outperformed PiT, and analysis of the results showed opportunities for substantial further performance improvement.