Prioritized Variable-length Test Cases Generation for Finite State Machines

TL;DR

This paper introduces a novel test generation strategy for finite state machines that handles specific testing constraints such as start/end states, prioritization, and variable path lengths, outperforming existing methods.

Contribution

The paper proposes a new approach for generating prioritized, variable-length test paths in FSMs that addresses practical testing needs not covered by previous strategies.

Findings

The new strategy outperforms the baseline in most configurations.

Three of six variants achieve the best results across different scenarios.

The approach is applicable to both functional and non-functional software testing.

Abstract

Model-based Testing (MBT) is an effective approach for testing when parts of a system-under-test have the characteristics of a finite state machine (FSM). Despite various strategies in the literature on this topic, little work exists to handle special testing situations. More specifically, when concurrently: (1) the test paths can start and end only in defined states of the FSM, (2) a prioritization mechanism that requires only defined states and transitions of the FSM to be visited by test cases is required, and (3) the test paths must be in a given length range, not necessarily of explicit uniform length. This paper presents a test generation strategy that satisfies all these requirements. A concurrent combination of these requirements is highly practical for real industrial testing. Six variants of possible algorithms to implement this strategy are described. Using a mixture of 180 problem instances from real automotive and defense projects and artificially generated FSMs, all variants are compared with a baseline strategy based on an established N-switch coverage concept modification. Various properties of the generated test paths and their potential to activate fictional defects defined in FSMs are evaluated. The presented strategy outperforms the baseline in most problem configurations. Out of the six analyzed variants, three give the best results even though a universal best performer is hard to identify. Depending on the application of the FSM, the strategy and evaluation presented in this paper are applicable both in testing functional and non-functional software requirements.