Reverse N-Wise Output-Oriented Testing for AI/ML and Quantum Computing Systems

TL;DR

This paper presents reverse n-wise output testing, a novel approach for systematically generating test inputs for AI/ML and quantum systems by covering output equivalence classes and behavior signatures, improving fault detection and validation.

Contribution

It introduces a mathematically grounded inversion paradigm that constructs covering arrays over output properties and solves inverse problems via metaheuristics for opaque models.

Findings

Enhanced fault detection in ML calibration and quantum errors

Improved test suite efficiency and coverage guarantees

Automated discovery of behavioral partitions and coverage drift monitoring

Abstract

Artificial intelligence/machine learning (AI/ML) systems and emerging quantum computing software present unprecedented testing challenges characterized by high-dimensional/continuous input spaces, probabilistic/non-deterministic output distributions, behavioral correctness defined exclusively over observable prediction behaviors and measurement outcomes, and critical quality dimensions, trustworthiness, fairness, calibration, robustness, error syndrome patterns, that manifest through complex multi-way interactions among semantically meaningful output properties rather than deterministic input-output mappings. This paper introduces reverse n-wise output testing, a mathematically principled paradigm inversion that constructs covering arrays directly over domain-specific output equivalence classes, ML confidence calibration buckets, decision boundary regions, fairness partitions, embedding clusters, ranking stability bands, quantum measurement outcome distributions (0-dominant, 1-dominant, superposition collapse), error syndrome patterns (bit-flip, phase-flip, correlated errors), then solves the computationally challenging black-box inverse mapping problem via gradient-free metaheuristic optimization to synthesize input feature configurations or quantum circuit parameters capable of eliciting targeted behavioral signatures from opaque models. The framework delivers synergistic benefits across both domains: explicit customer-centric prediction/measurement coverage guarantees, substantial improvements in fault detection rates for ML calibration/boundary failures and quantum error syndromes, enhanced test suite efficiency, and structured MLOps/quantum validation pipelines with automated partition discovery from uncertainty analysis and coverage drift monitoring.