Efficient quantum recurrent reinforcement learning via quantum reservoir computing

TL;DR

This paper introduces a quantum reinforcement learning framework using fixed, randomly initialized quantum LSTM reservoirs trained with the A3C algorithm, achieving efficient learning with reduced training complexity.

Contribution

It proposes a novel QRL approach utilizing fixed quantum LSTM reservoirs, simplifying training while maintaining competitive performance.

Findings

The QLSTM-Reservoir RL performs comparably to fully trained QLSTM models.

The approach reduces training complexity and computational cost.

Numerical simulations validate the effectiveness of the proposed framework.

Abstract

Quantum reinforcement learning (QRL) has emerged as a framework to solve sequential decision-making tasks, showcasing empirical quantum advantages. A notable development is through quantum recurrent neural networks (QRNNs) for memory-intensive tasks such as partially observable environments. However, QRL models incorporating QRNN encounter challenges such as inefficient training of QRL with QRNN, given that the computation of gradients in QRNN is both computationally expensive and time-consuming. This work presents a novel approach to address this challenge by constructing QRL agents utilizing QRNN-based reservoirs, specifically employing quantum long short-term memory (QLSTM). QLSTM parameters are randomly initialized and fixed without training. The model is trained using the asynchronous advantage actor-aritic (A3C) algorithm. Through numerical simulations, we validate the efficacy of our QLSTM-Reservoir RL framework. Its performance is assessed on standard benchmarks, demonstrating comparable results to a fully trained QLSTM RL model with identical architecture and training settings.