QEMI: A Quantum Software Stacks Testing Framework via Equivalence Modulo Inputs

TL;DR

QEMI introduces a novel testing framework for quantum software stacks using equivalence modulo inputs, effectively detecting bugs by comparing variants of quantum programs with dead code removal.

Contribution

This work adapts the classical EMI testing technique to quantum software, including a quantum program generator and dead code removal, to improve bug detection in quantum stacks.

Findings

Successfully identified 11 crash bugs in quantum software.

Detected 1 behavioral inconsistency across tested frameworks.

Expanded testing techniques for quantum software stacks.

Abstract

As quantum algorithms and hardware continue to evolve, ensuring the correctness of the quantum software stack (QSS) has become increasingly important. However, testing QSSes remains challenging due to the oracle problem, i.e., the lack of a reliable ground truth for expected program behavior. Existing metamorphic testing approaches often rely on equivalent circuit transformations, backend modifications, or parameter tuning to address this issue. In this work, inspired by Equivalence Modulo Inputs (EMI), we propose Quantum EMI (QEMI), a new testing approach for QSSes. Our key contributions include: (1) a random quantum program generator that produces code with dead code based on quantum control-flow structures, and (2) an adaptation of the EMI technique from classical compiler testing to generate variants by removing dead code. By comparing the behavior of these variants, we can detect potential bugs in QSS implementations. We applied QEMI to Qiskit, Q#, and Cirq, and successfully identified 11 crash bugs and 1 behavioral inconsistency. QEMI expands the limited set of testing techniques available for quantum software stacks by going beyond structural transformations and incorporating semantics-preserving ones into quantum program analysis.