A Taxonomic Review of Adaptive Random Testing: Current Status, Classifications, and Issues

TL;DR

This paper reviews the current state, classifications, and issues of Adaptive Random Testing (ART), highlighting its potential and the need for further empirical validation to enhance software testing effectiveness.

Contribution

It provides a comprehensive classification and assessment of existing ART methods, identifying research gaps and future directions for empirical studies.

Findings

Significant research efforts have been made in ART.

ART improves failure detection over traditional RT.

Further empirical validation is needed for industry application.

Abstract

Random testing (RT) is a black-box software testing technique that tests programs by generating random test inputs. It is a widely used technique for software quality assurance, but there has been much debate by practitioners concerning its failure-detection effectiveness. RT is argued to be possibly less effective by some researchers as it does not utilize any information about the program under test. Efforts to mainly improve the failure-detection capability of RT, have led to the proposition of Adaptive Random Testing (ART). ART takes advantage of the location information of previous non-fault-detecting test cases to enhance effectiveness as compared to RT. The approach has gained popularity and has a large number of theoretical studies and methods that employ different notions. In this review, our goal is to provide an overview of existing ART studies. We classify all ART studies and assess existing ART methods for numeric programs with a focus on their motivation, strategy, and findings. The study also discusses several worthy avenues related to ART. The review uses 109 ART papers in several journals, workshops, and conference proceedings. The results of the review show that significant research efforts have been made towards the field of ART, however further empirical studies are still required to make the technique applicable in different test scenarios in order to impact on the industry.