Benchmarking quantum computers

TL;DR

This paper surveys the current state of quantum computer benchmarking, emphasizing the importance of good benchmarks for measuring progress towards useful quantum computations and identifying open research challenges.

Contribution

It provides a comprehensive overview of quantum benchmarking methods, discusses their roles, and highlights key open questions in the field.

Findings

Different benchmarks quantify various aspects of quantum performance.

Existing benchmarks vary in scope and effectiveness.

Open research questions remain in developing comprehensive benchmarks.

Abstract

The rapid pace of development in quantum computing technology has sparked a proliferation of benchmarks for assessing the performance of quantum computing hardware and software. Good benchmarks empower scientists, engineers, programmers, and users to understand a computing system's power, but bad benchmarks can misdirect research and inhibit progress. In this Perspective, we survey the science of quantum computer benchmarking. We discuss the role of benchmarks and benchmarking, and how good benchmarks can drive and measure progress towards the long-term goal of useful quantum computations, i.e., "quantum utility". We explain how different kinds of benchmark quantify the performance of different parts of a quantum computer, survey existing benchmarks, examine recent trends in benchmarking, and highlight important open research questions in this field.