Impact and Performance of Randomized Test-Generation using Prolog

TL;DR

This paper investigates how randomized test-generation strategies using Prolog affect test performance, analyzing their efficiency and comparing two different randomization approaches through theoretical and empirical methods.

Contribution

It introduces two novel strategies for adding randomization to Prolog-based test generation and analyzes their impact on test performance using Markov chain models.

Findings

Randomization influences the mean time to reach test cases.

The two strategies differ in efficiency and effectiveness.

Empirical results compare the performance of both approaches.

Abstract

We study randomized generation of sequences of test-inputs to a system using Prolog. Prolog is a natural fit to generate test-sequences that have complex logical inter-dependent structure. To counter the problems posed by a large (or infinite) set of possible tests, randomization is a natural choice. We study the impact that randomization in conjunction with SLD resolution have on the test performance. To this end, this paper proposes two strategies to add randomization to a test-generating program. One strategy works on top of standard Prolog semantics, whereas the other alters the SLD selection function. We analyze the mean time to reach a test-case, and the mean number of generated test-cases in the framework of Markov chains. Finally, we provide an additional empirical evaluation and comparison between both approaches. Under consideration in Theory and Practice of Logic Programming (TPLP).