QPack Scores: Quantitative performance metrics for application-oriented quantum computer benchmarking

TL;DR

QPack provides a comprehensive set of quantitative metrics for benchmarking application-oriented quantum computers and simulators, enabling effective comparison of performance across different platforms in the NISQ era.

Contribution

This paper introduces QPack, a novel benchmarking suite with defined scores for runtime, accuracy, scalability, and capacity, specifically designed for quantum application performance evaluation.

Findings

QPack scores enable comparison of quantum hardware and simulators.

Application-oriented benchmarks reflect real-world quantum computing performance.

Initial data collected from IBMQ Nairobi illustrates practical benchmarking use.

Abstract

This paper presents the benchmark score definitions of QPack, an application-oriented cross-platform benchmarking suite for quantum computers and simulators, which makes use of scalable Quantum Approximate Optimization Algorithm and Variational Quantum Eigensolver applications. Using a varied set of benchmark applications, an insight of how well a quantum computer or its simulator performs on a general NISQ-era application can be quantitatively made. This paper presents what quantum execution data can be collected and transformed into benchmark scores for application-oriented quantum benchmarking. Definitions are given for an overall benchmark score, as well as sub-scores based on runtime, accuracy, scalability and capacity performance. Using these scores, a comparison is made between various quantum computer simulators, running both locally and on vendors' remote cloud services. We also use the QPack benchmark to collect a small set of quantum execution data of the IBMQ Nairobi quantum processor. The goal of the QPack benchmark scores is to give a holistic insight into quantum performance and the ability to make easy and quick comparisons between different quantum computers