Challenges for Reinforcement Learning in Quantum Circuit Design

TL;DR

This paper explores the use of reinforcement learning to optimize quantum circuit design within the constraints of NISQ-era quantum computing, introducing a new framework and benchmarking current RL algorithms.

Contribution

It formalizes quantum circuit design as a Markov decision process and introduces qcd-gym, a framework for learning policies to control quantum gates.

Findings

Benchmark comparisons reveal strengths and weaknesses of current RL algorithms.

Qcd-gym enables learning policies for controlling parameterized quantum gates.

The approach addresses challenges in quantum circuit design for NISQ devices.

Abstract

Quantum computing (QC) in the current NISQ era is still limited in size and precision. Hybrid applications mitigating those shortcomings are prevalent to gain early insight and advantages. Hybrid quantum machine learning (QML) comprises both the application of QC to improve machine learning (ML) and ML to improve QC architectures. This work considers the latter, leveraging reinforcement learning (RL) to improve quantum circuit design (QCD), which we formalize by a set of generic objectives. Furthermore, we propose qcd-gym, a concrete framework formalized as a Markov decision process, to enable learning policies capable of controlling a universal set of continuously parameterized quantum gates. Finally, we provide benchmark comparisons to assess the shortcomings and strengths of current state-of-the-art RL algorithms.