Hybrid Action Reinforcement Learning for Quantum Architecture Search

TL;DR

This paper introduces HyRLQAS, a hybrid reinforcement learning framework that jointly optimizes quantum circuit structure and parameters, leading to more efficient and accurate quantum algorithms for molecular ground-state energy estimation.

Contribution

It presents a unified hybrid-action RL approach for quantum architecture search that outperforms existing methods in accuracy and efficiency.

Findings

Achieves chemical-accuracy-level energy errors (~1e-8)

Reduces the number of gates needed for convergence

Enhances the efficiency of quantum circuit design

Abstract

Reinforcement learning-based Quantum Architecture Search (QAS) offers a promising avenue for automating the design of variational quantum circuits, but existing methods typically decouple discrete structure search from continuous parameter optimization, resulting in inefficient or brittle solutions. We propose HyRLQAS (Hybrid-Action Reinforcement Learning for Quantum Architecture Search), a unified reinforcement learning framework that jointly learns gate placement and parameter initialization within a hybrid discrete-continuous action space, while enabling dynamic refinement of previously placed gates. Trained in a variational quantum eigensolver setting, the agent constructs circuits that directly optimize molecular ground-state energies. Across multiple molecular benchmarks, HyRLQAS demonstrates strong and competitive performance against state-of-the-art QAS methods, achieving lower energy errors with fewer gates. Notably, HyRLQAS reaches chemical-accuracy-level convergence down to 1e-8 energy error after classical optimization, and policy-guided initialization reduces the iteration count of downstream classical optimizers. These results demonstrate that hybrid-action reinforcement learning provides a principled and effective mechanism for coupling circuit topology design with optimization-aware parameterization.