Bugs in Quantum Computing Platforms: An Empirical Study

TL;DR

This study analyzes 223 real-world bugs in quantum computing platforms, revealing that nearly 40% are quantum-specific, often manifesting as unexpected outputs, and introduces a hierarchy of bug patterns to aid developers.

Contribution

It provides the first comprehensive empirical analysis of bugs in quantum platforms, identifying quantum-specific bugs and proposing a hierarchy of bug patterns for improved debugging.

Findings

39.9% of bugs are quantum-specific

Quantum bugs often cause unexpected outputs

Introduces a hierarchy of 10 quantum-specific bug patterns

Abstract

The interest in quantum computing is growing, and with it, the importance of software platforms to develop quantum programs. Ensuring the correctness of such platforms is important, and it requires a thorough understanding of the bugs they typically suffer from. To address this need, this paper presents the first in-depth study of bugs in quantum computing platforms. We gather and inspect a set of 223 real-world bugs from 18 open-source quantum computing platforms. Our study shows that a significant fraction of these bugs (39.9%) are quantum-specific, calling for dedicated approaches to prevent and find them. The bugs are spread across various components, but quantum-specific bugs occur particularly often in components that represent, compile, and optimize quantum programming abstractions. Many quantum-specific bugs manifest through unexpected outputs, rather than more obvious signs of misbehavior, such as crashes. Finally, we present a hierarchy of recurrent bug patterns, including ten novel, quantum-specific patterns. Our findings not only show the importance and prevalence bugs in quantum computing platforms, but they help developers to avoid common mistakes and tool builders to tackle the challenge of preventing, finding, and fixing these bugs.