Mining Precise Test Oracle Modelled by FSM

TL;DR

This paper presents a test-driven method using non-deterministic FSMs to help experts select the most accurate test oracle among multiple candidates for reactive systems.

Contribution

It introduces a novel approach that employs non-deterministic FSMs and Boolean partitioning to assist experts in choosing precise test oracles from ambiguous system requirements.

Findings

Empirical evaluation shows the approach's effectiveness.

The method aids experts in oracle selection process.

Partitioning reduces candidate sets efficiently.

Abstract

Precise test oracles for reactive systems such as critical control systems and communication protocols can be modelled with deterministic finite state machines (FSMs). Among other roles, they serve in evaluating the correctness of systems under test. A great number of candidate precise oracles (shortly, candidates) can be produced at the system design phase due to uncertainties, e.g., when interpreting their requirements expressed in ambiguous natural languages. Selecting the proper candidate becomes challenging for an expert. We propose a test-driven approach to assist experts in this selection task. The approach uses a non deterministic FSM to represent the candidates, includes the partitioning of the candidates into subsets of candidates via Boolean encodings and requires the intervention of experts to select subsets. We perform an empirical evaluation of the applicability of the proposed approach.