Application-Oriented Benchmarking of Quantum Generative Learning Using QUARK

TL;DR

This paper extends the QUARK benchmarking framework to evaluate quantum generative models across various architectures, datasets, and hardware, providing standardized metrics for assessing their performance and generalization.

Contribution

The paper introduces new extensions to the QUARK framework for benchmarking quantum generative models, including evaluation on hardware and novel metrics for data quality.

Findings

Quantum generative models can be trained with different ansatzes and datasets.

Models evaluated on GPU and quantum hardware show comparable performance.

Broad metrics reveal insights into the generalization and novelty of generated data.

Abstract

Benchmarking of quantum machine learning (QML) algorithms is challenging due to the complexity and variability of QML systems, e.g., regarding model ansatzes, data sets, training techniques, and hyper-parameters selection. The QUantum computing Application benchmaRK (QUARK) framework simplifies and standardizes benchmarking studies for quantum computing applications. Here, we propose several extensions of QUARK to include the ability to evaluate the training and deployment of quantum generative models. We describe the updated software architecture and illustrate its flexibility through several example applications: (1) We trained different quantum generative models using several circuit ansatzes, data sets, and data transformations. (2) We evaluated our models on GPU and real quantum hardware. (3) We assessed the generalization capabilities of our generative models using a broad set of metrics that capture, e.g., the novelty and validity of the generated data.