A methodology for comparing and benchmarking quantum devices

TL;DR

This paper proposes a standardized framework for defining, articulating, and justifying success criteria and benchmarks in quantum device development, addressing the current lack of consistency and standards in the field.

Contribution

It introduces a methodology for comparing and benchmarking quantum devices, enabling clearer evaluation criteria across research and industry.

Findings

Provides a structured framework for success criteria

Facilitates consistent benchmarking of quantum devices

Supports clearer communication of quantum device performance

Abstract

Quantum Computing (QC) is undergoing a high rate of development, investment and research devoted to its improvement.However, there is little consensus in the industry and wider literature as to what improvement might consist of beyond ambiguous statements of "more qubits" and "fewer errors". Before one can decide how to improve something, it is first necessary to define the criteria for success: what are the metrics or statistics that are relevant to the problem? The lack of clarity surrounding this question has led to a rapidly developing capability with little consistency or standards present across the board. This paper lays out a framework by which any user, developer or researcher can define, articulate and justify the success criteria and associated benchmarks that have been used to solve their problem or make their claim.