Quantum deep recurrent reinforcement learning

TL;DR

This paper introduces a quantum recurrent neural network-based reinforcement learning agent using quantum LSTM, demonstrating improved stability and performance over classical methods in benchmark tasks through numerical simulations.

Contribution

It presents a novel quantum RL framework with quantum LSTM, advancing the training of quantum agents in partially observable environments.

Findings

QLSTM-DRQN outperforms classical DRQN in Cart-Pole tasks

Quantum RL agents show more stable learning curves

Numerical simulations validate the effectiveness of quantum recurrent networks

Abstract

Recent advances in quantum computing (QC) and machine learning (ML) have drawn significant attention to the development of quantum machine learning (QML). Reinforcement learning (RL) is one of the ML paradigms which can be used to solve complex sequential decision making problems. Classical RL has been shown to be capable to solve various challenging tasks. However, RL algorithms in the quantum world are still in their infancy. One of the challenges yet to solve is how to train quantum RL in the partially observable environments. In this paper, we approach this challenge through building QRL agents with quantum recurrent neural networks (QRNN). Specifically, we choose the quantum long short-term memory (QLSTM) to be the core of the QRL agent and train the whole model with deep $Q$-learning. We demonstrate the results via numerical simulations that the QLSTM-DRQN can solve standard benchmark such as Cart-Pole with more stable and higher average scores than classical DRQN with similar architecture and number of model parameters.