Complexity of adaptive testing in scenarios defined extensionally

TL;DR

This paper investigates the computational complexity of adaptive testing strategies in extensionally defined scenarios, analyzing four variants and establishing their PSPACE-completeness and Log-APX-hardness.

Contribution

It provides a complexity analysis of adaptive testing in extensionally defined settings, identifying key hardness results for four problem variants.

Findings

Complexity varies across problem variants.

PSPACE-completeness established for certain variants.

Log-APX-hardness demonstrated for others.

Abstract

In this paper we consider a testing setting where the set of possible definitions of the Implementation Under Test (IUT), as well as the behavior of each of these definitions in all possible interactions, are extensionally defined, i.e., on an element-by-element and case-by-case basis. Under this setting, the problem of finding the minimum testing strategy such that collected observations will necessarily let us decide whether the IUT is correct or not (i.e., whether it necessarily belongs to the set of possible correct definitions or not) is studied in four possible problem variants: with or without non-determinism; and with or without more than one possible definition in the sets of possible correct and incorrect definitions. The computational complexity of these variants is studied, and properties such as PSPACE-completeness and Log-APX-hardness are identified.