BenchRL-QAS: Benchmarking reinforcement learning algorithms for quantum architecture search

TL;DR

BenchRL-QAS introduces a comprehensive benchmarking framework for evaluating reinforcement learning algorithms across various quantum architecture search tasks, revealing performance dependencies on task specifics and noise conditions.

Contribution

It provides the first extensive benchmark for RL in quantum architecture search, comparing multiple algorithms across diverse quantum tasks and noise settings.

Findings

No single RL method dominates across all tasks.

Performance varies with task type, qubit count, and noise.

RL-based VQC can outperform baseline VQCs.

Abstract

We present BenchRL-QAS, a unified benchmarking framework for reinforcement learning (RL) in quantum architecture search (QAS) across a spectrum of variational quantum algorithm tasks on 2- to 8-qubit systems. Our study systematically evaluates 9 different RL agents, including both value-based and policy-gradient methods, on quantum problems such as variational eigensolver, quantum state diagonalization, variational quantum classification (VQC), and state preparation, under both noiseless and noisy execution settings. To ensure fair comparison, we propose a weighted ranking metric that integrates accuracy, circuit depth, gate count, and training time. Results demonstrate that no single RL method dominates universally, the performance dependents on task type, qubit count, and noise conditions providing strong evidence of no free lunch principle in RL-QAS. As a byproduct we observe that a carefully chosen RL algorithm in RL-based VQC outperforms baseline VQCs. BenchRL-QAS establishes the most extensive benchmark for RL-based QAS to date, codes and experimental made publicly available for reproducibility and future advances.