Effective grey-box testing with partial FSM models

TL;DR

This paper introduces a new conformance relation called strong reduction for partial nondeterministic finite state machines, enabling efficient grey-box testing with significantly reduced test suite sizes.

Contribution

It presents a novel strong reduction relation and a complete test generation algorithm that leverages grey-box information to minimize test suite size for model-based testing.

Findings

Test suite size is linear in the model's state space.

Grey-box testing reduces test suite size compared to black-box testing.

The method is effective for systems with state-dependent input enabling.

Abstract

For partial, nondeterministic, finite state machines, a new conformance relation called strong reduction is presented. It complements other existing conformance relations in the sense that the new relation is well-suited for model-based testing of systems whose inputs are enabled or disabled, depending on the actual system state. Examples of such systems are graphical user interfaces and systems with interfaces that can be enabled or disabled in a mechanical way. We present a new test generation algorithm producing complete test suites for strong reduction. The suites are executed according to the grey-box testing paradigm: it is assumed that the state-dependent sets of enabled inputs can be identified during test execution, while the implementation states remain hidden, as in black-box testing. It is shown that this grey-box information is exploited by the generation algorithm in such a way that the resulting best-case test suite size is only linear in the state space size of the reference model. Moreover, examples show that this may lead to significant reductions of test suite size in comparison to true black-box testing for strong reduction.