Scalable Benchmarks for Gate-Based Quantum Computers

TL;DR

This paper introduces a scalable, hardware-agnostic benchmarking framework for quantum computers, enabling intuitive and visual comparison of device performance across multiple metrics relevant to real-world applications.

Contribution

It develops a set of six structured, scalable tests that assess quantum device performance in a way that is intuitive, visual, and applicable to current and future hardware.

Findings

Benchmark results for 21 quantum devices from IBM, Rigetti, and IonQ.

Framework effectively differentiates device performance based on real-life relevant metrics.

Tests are scalable and suitable for future quantum hardware evaluation.

Abstract

In the near-term "NISQ"-era of noisy, intermediate-scale, quantum hardware and beyond, reliably determining the quality of quantum devices becomes increasingly important: users need to be able to compare them with one another, and make an estimate whether they are capable of performing a given task ahead of time. In this work, we develop and release an advanced quantum benchmarking framework in order to help assess the state of the art of current quantum devices. Our testing framework measures the performance of universal quantum devices in a hardware-agnostic way, with metrics that are aimed to facilitate an intuitive understanding of which device is likely to outperform others on a given task. This is achieved through six structured tests that allow for an immediate, visual assessment of how devices compare. Each test is designed with scalability in mind, making this framework not only suitable for testing the performance of present-day quantum devices, but also of those released in the foreseeable future. The series of tests are motivated by real-life scenarios, and therefore emphasise the interplay between various relevant characteristics of quantum devices, such as qubit count, connectivity, and gate and measurement fidelity. We present the benchmark results of twenty-one different quantum devices from IBM, Rigetti and IonQ.