Comparative Study of Quantum Transpilers: Evaluating the Performance of qiskit-braket-provider, qBraid-SDK, and Pytket Extensions

TL;DR

This paper evaluates the performance of SDK-to-SDK quantum transpilers, introducing tools for unbiased benchmarking, and finds that qiskit-braket-provider outperforms others in success rate and speed, guiding future development.

Contribution

It presents a comprehensive benchmarking framework and compares leading quantum transpilers, highlighting strengths and limitations of each, especially the superior performance of qiskit-braket-provider.

Findings

qiskit-braket-provider has a 0.2% failure rate.

qBraid-SDK offers robust multi-SDK transpilation but is slower.

pytket extensions are fast but limited with complex circuits.

Abstract

In this study, we present a comprehensive evaluation of popular SDK-to-SDK quantum transpilers (that is transpilers that takes a quantum circuit from an initial SDK and output a quantum circuit in another SDK), focusing on critical metrics such as correctness, failure rate, and transpilation time. To ensure unbiased evaluation and accommodate diverse quantum computing scenarios, we developed two dedicated tools: RandomQC, for generating random quantum circuits across various types (pure random, VQE-like, and SDK-specific circuits), and Benchmarq, to streamline the benchmarking process. Using these tools, we benchmarked prominent quantum transpilers as of February 2024. Our results highlight the superior performance of the qiskit-braket-provider, a specialized transpiler from Qiskit to Braket, achieving a remarkably low failure rate of 0.2%. The qBraid-SDK, offering generalized transpilation across multiple SDKs, demonstrated robust but slower performance. The pytket extensions, while fast, faced limitations with complex circuits due to their one-to-one transpilation approach. In particular, the exceptional performance of the qiskit-bracket-provider stems not only from its specialization but also from its architecture, which combines one-to-one transpilation with gate decomposition for unsupported gates, enhancing both speed and capability. This study aims to provide practical guidelines to users of SDK-to-SDK quantum transpilers and guidance to developers for improving the design and development of future tools.