Quantum enhancements for deep reinforcement learning in large spaces

TL;DR

This paper explores how quantum algorithms can enhance deep reinforcement learning, especially in large spaces, by leveraging energy-based models and quantum speed-ups to improve learning efficiency and performance.

Contribution

It introduces quantum-enhanced neural network models for reinforcement learning, demonstrating advantages in complex environments and proposing methods to leverage quantum algorithms for efficiency.

Findings

Quantum models with sampling bottlenecks outperform classical counterparts in complex tasks.

Energy-based models and quantum algorithms can trade off learning performance for computational efficiency.

Proposed quantum methods can accelerate classical sampling, improving overall learning speed.

Abstract

In the past decade, the field of quantum machine learning has drawn significant attention due to the prospect of bringing genuine computational advantages to now widespread algorithmic methods. However, not all domains of machine learning have benefited equally from quantum enhancements. Notably, deep learning and reinforcement learning, despite their tremendous success in the classical domain, both individually and combined, remain relatively unaddressed by the quantum community. Arguably, one reason behind this is the systematic use in these domains of models and methods without prominent computational bottlenecks, leaving little room for quantum improvements. In this work, we study the state-of-the-art neural-network approaches for reinforcement learning with quantum enhancements in mind. We demonstrate the substantial learning advantage that models with a sampling bottleneck can provide over conventional neural network architectures in complex learning environments. These so-called energy-based models, like deep energy-based reinforcement learning, and deep projective simulation that we also introduce in this work, effectively allow to trade off learning performance for efficiency of computation. To alleviate the additional computational costs, we propose to leverage future and near-term quantum algorithms, resulting in overall more advantageous learning algorithms. This is achieved using cutting-edge and new quantum computing machinery to speed-up classical sampling methods and by employing generalized models to gain an additional quantum advantage.