METFORD -- Mutation tEsTing Framework fOR anDroid

TL;DR

This paper introduces METFORD, a mutation testing framework for Android that reduces computational costs by using mutant schemata, achieving faster execution, lower resource usage, and reduced carbon footprint compared to traditional methods.

Contribution

It presents a novel Android mutation testing framework utilizing mutant schemata, significantly improving efficiency and sustainability over traditional mutation testing approaches.

Findings

Mutant schemata reduce mutation testing time by 8.50%.

They require 99.78% less disk space.

They are 6.45% faster and have an 8.18% lower carbon footprint.

Abstract

Mutation testing may be used to guide test case generation and as a technique to assess the quality of test suites. Despite being used frequently, mutation testing is not so commonly applied in the mobile world. One critical challenge in mutation testing is dealing with its computational cost. Generating mutants, running test cases over each mutant, and analyzing the results may require significant time and resources. This research aims to contribute to reducing Android mutation testing costs. It implements mutation testing operators (traditional and Android-specific) according to mutant schemata (implementing multiple mutants into a single code file). It also describes an Android mutation testing framework developed to execute test cases and determine mutation scores. Additional mutation operators can be implemented in JavaScript and easily integrated into the framework. The overall approach is validated through case studies showing that mutant schemata have advantages over the traditional mutation strategy (one file per mutant). The results show mutant schemata overcome traditional mutation in all evaluated aspects with no additional cost: it takes 8.50% less time for mutant generation, requires 99.78% less disk space, and runs, on average, 6.45% faster than traditional mutation. Moreover, considering sustainability metrics, mutant schemata have 8,18% less carbon footprint than traditional strategy.