QuCAT: A Combinatorial Testing Tool for Quantum Software

TL;DR

QuCAT is a new tool for systematic combinatorial testing of quantum software, generating test suites of varying strength to efficiently detect faults in quantum programs.

Contribution

It introduces a quantum software testing tool that automates combinatorial test generation and fault detection, with evaluation on real quantum programs.

Findings

Low-strength test suites detect faults at low cost

Higher-strength test suites improve fault detection for difficult faults

The tool demonstrates effectiveness and efficiency in quantum program testing

Abstract

With the increased developments in quantum computing, the availability of systematic and automatic testing approaches for quantum programs is becoming increasingly essential. To this end, we present the quantum software testing tool QuCAT for combinatorial testing of quantum programs. QuCAT provides two functionalities of use. With the first functionality, the tool generates a test suite of a given strength (e.g., pair-wise). With the second functionality, it generates test suites with increasing strength until a failure is triggered or a maximum strength is reached. QuCAT uses two test oracles to check the correctness of test outputs. We assess the cost and effectiveness of QuCAT with 3 faulty versions of 5 quantum programs. Results show that combinatorial test suites with a low strength can find faults with limited cost, while a higher strength performs better to trigger some difficult faults with relatively higher cost. Repository: https://github.com/Simula-COMPLEX/qucat-tool Video: https://youtu.be/UsqgOudKLio