Assessing student learning in quantum computing: The challenging case of phase kickback

TL;DR

This paper discusses the development and refinement of an assessment item for quantum computing education, highlighting challenges in probing student understanding of quantum concepts.

Contribution

It provides a detailed case study of creating a reliable assessment question for quantum computing, emphasizing the importance of diverse data sources.

Findings

Student reasoning in quantum computing is complex and requires careful assessment.

Triangulating multiple data sources improves the reliability of assessment items.

The development process reveals insights into student misconceptions in quantum concepts.

Abstract

A major challenge for quantum workforce development is the need to both understand and reliably assess student learning of quantum information science (QIS) fundamentals. Yet student thinking is notoriously difficult to probe, even for seasoned education researchers. This article presents the story of Item 15 on the Quantum Computing Conceptual Survey (QCCS). This assessment item underwent more revision and discussion within the team than the remaining 19 assessment questions combined. This paper provides a behind-the-scenes look at the development of this assessment question: a story that both reveals interesting findings about student reasoning in quantum computing and illustrates why quantum education researchers insist on triangulating diverse quantitative and qualitative data sources when developing and refining assessment items, with implications for any researcher looking to understand and measure student conceptual reasoning in quantum computing, as well as for QIS curriculum and workforce development more broadly.