Reinforcement Learning vs. Gradient-Based Optimisation for Robust Energy Landscape Control of Spin-1/2 Quantum Networks

TL;DR

This paper compares reinforcement learning and gradient-based optimization for controlling quantum spin networks, showing RL's superior robustness in noisy, challenging scenarios.

Contribution

It demonstrates the effectiveness of policy gradient methods in quantum control, especially under noise, outperforming traditional gradient-based approaches.

Findings

Reinforcement learning handles noisy quantum control problems better.

Controllers found by RL are more robust to Hamiltonian noise.

Gradient-based methods struggle with noisy, complex quantum landscapes.

Abstract

We explore the use of policy gradient methods in reinforcement learning for quantum control via energy landscape shaping of XX-Heisenberg spin chains in a model agnostic fashion. Their performance is compared to finding controllers using gradient-based L-BFGS optimisation with restarts, with full access to an analytical model. Hamiltonian noise and coarse-graining of fidelity measurements are considered. Reinforcement learning is able to tackle challenging, noisy quantum control problems where L-BFGS optimization algorithms struggle to perform well. Robustness analysis under different levels of Hamiltonian noise indicates that controllers found by reinforcement learning appear to be less affected by noise than those found with L-BFGS.