Benchmarking the performance of quantum computing software

TL;DR

Benchpress is an open-source benchmarking suite that evaluates the performance and functionality of various quantum computing software packages using over 1000 tests on circuits with up to 930 qubits, aiding future hardware development.

Contribution

The paper introduces Benchpress, a comprehensive, flexible benchmarking framework for quantum software, enabling standardized performance assessment across multiple platforms.

Findings

Performance metrics for quantum software packages are systematically measured.

Benchpress can evaluate circuits with up to 930 qubits and 10^6 gates.

The framework is adaptable to future quantum hardware architectures.

Abstract

We present Benchpress, a benchmarking suite for evaluating the performance and range of functionality of multiple quantum computing software development kits. This suite consists of a collection of over $1000$ tests measuring key performance metrics for a wide variety of operations on quantum circuits comprised of up to $930$ qubits and $\mathcal{O}(10^{6})$ two-qubit gates, as well as an execution framework for running the tests over multiple quantum software packages in a unified manner. We give a detailed overview of the benchmark suite, its methodology, and generate representative results over seven different quantum software packages. The flexibility of the Benchpress framework allows for benchmarking that not only keeps pace with quantum hardware improvements but can preemptively gauge the quantum circuit processing costs of future device architectures. Being open-source, Benchpress ensures the transparency and verification of performance claims.