Model-Free Quantum Control with Reinforcement Learning

TL;DR

This paper introduces a model-free reinforcement learning approach for quantum control that learns control policies directly from measurement outcomes, eliminating model bias and improving sample efficiency in quantum systems.

Contribution

The authors develop a circuit-based reinforcement learning method for quantum control that does not rely on system models, enabling adaptive and efficient control in experimental quantum platforms.

Findings

Outperforms existing model-free methods in sample efficiency

Successfully trains agents to prepare non-classical states and execute quantum gates

Applicable to superconducting circuits and trapped ion systems

Abstract

Model bias is an inherent limitation of the current dominant approach to optimal quantum control, which relies on a system simulation for optimization of control policies. To overcome this limitation, we propose a circuit-based approach for training a reinforcement learning agent on quantum control tasks in a model-free way. Given a continuously parameterized control circuit, the agent learns its parameters through trial-and-error interaction with the quantum system, using measurement outcomes as the only source of information about the quantum state. Focusing on control of a harmonic oscillator coupled to an ancilla qubit, we show how to reward the learning agent using measurements of experimentally available observables. We train the agent to prepare various non-classical states using both unitary control and control with adaptive measurement-based quantum feedback, and to execute logical gates on encoded qubits. This approach significantly outperforms widely used model-free methods in terms of sample efficiency. Our numerical work is of immediate relevance to superconducting circuits and trapped ions platforms where such training can be implemented in experiment, allowing complete elimination of model bias and the adaptation of quantum control policies to the specific system in which they are deployed.