Bet and Run for Test Case Generation

TL;DR

This paper explores the application of the Bet and Run restart strategy, originally designed for theoretical search problems, to test case generation, but finds limited improvements with existing parameters.

Contribution

It adapts and evaluates the Bet and Run strategy for test case generation, highlighting its performance limitations with current parameters.

Findings

Bet and Run does not significantly improve test case quality with existing parameters.

The strategy's effectiveness varies depending on the problem instance.

Further parameter tuning may be necessary for better results.

Abstract

Anyone working in the technology sector is probably familiar with the question: "Have you tried turning it off and on again?", as this is usually the default question asked by tech support. Similarly, it is known in search based testing that metaheuristics might get trapped in a plateau during a search. As a human, one can look at the gradient of the fitness curve and decide to restart the search, so as to hopefully improve the results of the optimization with the next run. Trying to automate such a restart, it has to be programmatically decided whether the metaheuristic has encountered a plateau yet, which is an inherently difficult problem. To mitigate this problem in the context of theoretical search problems, the Bet and Run strategy was developed, where multiple algorithm instances are started concurrently, and after some time all but the single most promising instance in terms of fitness values are killed. In this paper, we adopt and evaluate the Bet and Run strategy for the problem of test case generation. Our work indicates that use of this restart strategy does not generally lead to gains in the quality metrics, when instantiated with the best parameters found in the literature.