Quantum Testing in the Wild: A Case Study with Qiskit Algorithms

TL;DR

This paper empirically analyzes testing practices in quantum algorithms within Qiskit, identifying common patterns and highlighting the dominance of classical testing methods in quantum software engineering.

Contribution

It systematically categorizes testing patterns for quantum algorithms and compares them with recent research, revealing gaps in adoption of advanced quantum testing techniques.

Findings

Seven distinct testing patterns identified in Qiskit algorithms

Classical testing techniques dominate quantum software testing

Research advances in quantum testing are not widely adopted in practice

Abstract

Although classical computing has excelled in a wide range of applications, there remain problems that push the limits of its capabilities, especially in fields like cryptography, optimization, and materials science. Quantum computing introduces a new computational paradigm, based on principles of superposition and entanglement to explore solutions beyond the capabilities of classical computation. With the increasing interest in the field, there are challenges and opportunities for academics and practitioners in terms of software engineering practices, particularly in testing quantum programs. This paper presents an empirical study of testing patterns in quantum algorithms. We analyzed all the tests handling quantum aspects of the implementations in the Qiskit Algorithms library and identified seven distinct patterns that make use of (1) fixed seeds for algorithms based on random elements; (2) deterministic oracles; (3) precise and approximate assertions; (4) Data-Driven Testing (DDT); (5) functional testing; (6) testing for intermediate parts of the algorithms being tested; and (7) equivalence checking for quantum circuits. Our results show a prevalence of classical testing techniques to test the quantum-related elements of the library, while recent advances from the research community have yet to achieve wide adoption among practitioners.