Quantum Circuit Mutants: Empirical Analysis and Recommendations

TL;DR

This paper presents an extensive empirical study on quantum circuit mutants, analyzing how circuit features and mutation types influence mutant detection, and offers a tool for generating tailored benchmarks to improve quantum software testing.

Contribution

It introduces a large-scale empirical evaluation of quantum circuit mutants, providing insights and a tool for systematic mutant generation based on circuit characteristics.

Findings

Over 700K faulty quantum circuit benchmarks analyzed

Insights into how circuit features affect mutant detection

A tool for recommending mutants based on user-specified attributes

Abstract

As a new research area, quantum software testing lacks systematic testing benchmarks to assess testing techniques' effectiveness. Recently, some open-source benchmarks and mutation analysis tools have emerged. However, there is insufficient evidence on how various quantum circuit characteristics (e.g., circuit depth, number of quantum gates), algorithms (e.g., Quantum Approximate Optimization Algorithm), and mutation characteristics (e.g., mutation operators) affect the detection of mutants in quantum circuits. Studying such relations is important to systematically design faulty benchmarks with varied attributes (e.g., the difficulty in detecting a seeded fault) to facilitate assessing the cost-effectiveness of quantum software testing techniques efficiently. To this end, we present a large-scale empirical evaluation with more than 700K faulty benchmarks (quantum circuits) generated by mutating 382 real-world quantum circuits. Based on the results, we provide valuable insights for researchers to define systematic quantum mutation analysis techniques. We also provide a tool to recommend mutants to users based on chosen characteristics (e.g., a quantum algorithm type) and the required difficulty of detecting mutants. Finally, we also provide faulty benchmarks that can already be used to assess the cost-effectiveness of quantum software testing techniques.