Evaluating Three Levels of Quantum Metrics on Quantum-Inspire Hardware

TL;DR

This paper categorizes quantum metrics into three levels—component, system, and application—and evaluates them on Quantum-Inspire's Starmon-5 device, providing a comprehensive benchmarking approach.

Contribution

It introduces a structured grouping of quantum metrics into three levels and applies this framework to benchmark a real quantum device comprehensively.

Findings

Metrics are grouped into component, system, and application levels.

The evaluation provides the most complete benchmark of a quantum device to date.

Insights into the merits and uses of different metric levels.

Abstract

With the rise of quantum computing, many quantum devices have been developed and many more devices are being developed as we speak. This begs the question of which device excels at which tasks and how to compare these different quantum devices with one another. The answer is given by quantum metrics, of which many exist today already. Different metrics focus on different aspects of (quantum) devices and choosing the right metric to benchmark one device against another is a difficult choice. In this paper we aim to give an overview of this zoo of metrics by grouping established metrics in three levels: component level, system level and application level. With this characterisation we also mention what the merits and uses are for each of the different levels. In addition, we evaluate these metrics on the Starmon-5 device of Quantum-Inspire through the cloud access, giving the most complete benchmark of a quantum device from an user experience to date.