A Snowballing Literature Study on Test Amplification

TL;DR

This paper surveys the emerging field of test amplification, which enhances manual tests by exploiting embedded knowledge to achieve various engineering goals beyond just coverage, through a systematic literature review.

Contribution

It provides the first comprehensive overview of test amplification research, identifying its goals, techniques, and the scope beyond coverage maximization.

Findings

Test amplification encompasses diverse engineering goals.

The field is coherent and evolving with multiple techniques.

The survey aids researchers and practitioners in understanding test amplification.

Abstract

The adoption of agile development approaches has put an increased emphasis on developer testing, resulting in software projects with strong test suites. These suites include a large number of test cases, in which developers embed knowledge about meaningful input data and expected properties in the form of oracles. This article surveys various works that aim at exploiting this knowledge in order to enhance these manually written tests with respect to an engineering goal (e.g., improve coverage of changes or increase the accuracy of fault localization). While these works rely on various techniques and address various goals, we believe they form an emerging and coherent field of research, which we call `test amplification'. We devised a first set of papers from DBLP, looking for all papers containing `test' and `amplification' in their title. We reviewed the 70 papers in this set and selected the 4 papers that fit our definition of test amplification. We use these 4 papers as the seed for our snowballing study, and systematically followed the citation graph. This study is the first that draws a comprehensive picture of the different engineering goals proposed in the literature for test amplification. In particular, we note that the goal of test amplification goes far beyond maximizing coverage only. We believe that this survey will help researchers and practitioners entering this new field to understand more quickly and more deeply the intuitions, concepts and techniques used for test amplification.