Code Comments for Quantum Software Development Kits: An Empirical Study on Qiskit

TL;DR

This paper presents CC4Q, a dataset of quantum code comments for Qiskit, and an empirical study revealing key differences and quantum-specific knowledge in comments compared to classical software.

Contribution

It introduces the first dataset of quantum code comments and provides an empirical analysis of their structure, developer intentions, and quantum-specific content.

Findings

Quantum comments differ significantly from classical ones.

Quantum-specific knowledge is prevalent in code comments.

The dataset facilitates future research in quantum software development.

Abstract

Quantum computing is gaining attention from academia and industry. With the quantum Software Development Kits (SDKs), programmers can develop quantum software to explore the power of quantum computing. However, programmers may face challenges in understanding quantum software due to the non-intuitive quantum mechanics. To facilitate software development and maintenance, code comments offered in quantum SDKs serve as a natural language explanation of program functionalities and logical flows. Despite their importance, scarce research systematically reports their value and provides constructive guidelines for programmers. To address this gap, our paper focuses on Qiskit, one of the most popular quantum SDKs, and presents CC4Q, the first dataset of code comments for quantum computing. CC4Q incorporates 9677 code comment pairs and 21970 sentence-level code comment units, the latter of which involve heavy human annotation. Regarding the annotation, we validate the applicability of the developer-intent taxonomy used in classical programs, and also propose a new taxonomy considering quantum-specific knowledge. We conduct an empirical study comprehensively interpreting code comments from three perspectives: comment structure and coverage, developers' intentions, and associated quantum topics. Our findings uncover key differences in code comments between classical and quantum software, and also outline quantum-specific knowledge relevant to quantum software development.