Testing Systems of Concurrent Black-boxes--an Automata-Theoretic and Decompositional Approach

TL;DR

This paper presents a novel automata-theoretic and decompositional approach for testing concurrent black-box systems, reducing the global testing problem to individual black-box testing without integration testing, and demonstrating efficiency and completeness.

Contribution

It introduces a new decompositional testing method that avoids integration testing, is automata-based, complete, sound, and automatable for large systems.

Findings

Significantly reduces the number of tests needed for large systems

The approach is complete and sound, ensuring reliable results

Experimental results show efficiency even with large 'Bad' sets

Abstract

The global testing problem studied in this paper is to seek a definite answer to whether a system of concurrent black-boxes has an observable behavior in a given finite (but could be huge) set "Bad". We introduce a novel approach to solve the problem that does not require integration testing. Instead, in our approach, the global testing problem is reduced to testing individual black-boxes in the system one by one in some given order. Using an automata-theoretic approach, test sequences for each individual black-box are generated from the system's description as well as the test results of black-boxes prior to this black-box in the given order. In contrast to the conventional compositional/modular verification/testing approaches, our approach is essentially decompositional. Also, our technique is complete, sound, and can be carried out automatically. Our experiment results show that the total number of tests needed to solve the global testing problem is substantially small even for an extremely large "Bad".