Mapping the Structure and Evolution of Software Testing Research Over the Past Three Decades

TL;DR

This paper maps three decades of software testing research, identifying key topics, their evolution, and emerging trends using co-word analysis to reveal the field's multidisciplinary nature and future directions.

Contribution

It provides a comprehensive network-based map of software testing research topics, their connections, and evolution over 30 years, highlighting emerging areas and declining topics.

Findings

16 high-level research topics identified

Emerging keywords include machine learning and mobile apps

Strong connections between creation guidance and automated test generation

Abstract

Background: The field of software testing is growing and rapidly-evolving. Aims: Based on keywords assigned to publications, we seek to identify predominant research topics and understand how they are connected and have evolved. Method: We apply co-word analysis to map the topology of testing research as a network where author-assigned keywords are connected by edges indicating co-occurrence in publications. Keywords are clustered based on edge density and frequency of connection. We examine the most popular keywords, summarize clusters into high-level research topics, examine how topics connect, and examine how the field is changing. Results: Testing research can be divided into 16 high-level topics and 18 subtopics. Creation guidance, automated test generation, evolution and maintenance, and test oracles have particularly strong connections to other topics, highlighting their multidisciplinary nature. Emerging keywords relate to web and mobile apps, machine learning, energy consumption, automated program repair and test generation, while emerging connections have formed between web apps, test oracles, and machine learning with many topics. Random and requirements-based testing show potential decline. Conclusions: Our observations, advice, and map data offer a deeper understanding of the field and inspiration regarding challenges and connections to explore.