A quantum-classical reinforcement learning model to play Atari games

TL;DR

This paper introduces a hybrid quantum-classical reinforcement learning model that successfully plays Atari games, demonstrating its potential for complex environments and providing insights into quantum-classical integration.

Contribution

It presents a novel hybrid quantum-classical model capable of playing Atari games, bridging the gap between simple benchmarks and complex real-world scenarios.

Findings

The hybrid model can solve Pong and achieve scores comparable to classical models in Breakout.

Hyperparameter settings significantly affect quantum-classical interplay.

The work advances understanding of near-term quantum reinforcement learning applications.

Abstract

Recent advances in reinforcement learning have demonstrated the potential of quantum learning models based on parametrized quantum circuits as an alternative to deep learning models. On the one hand, these findings have shown the ultimate exponential speed-ups in learning that full-blown quantum models can offer in certain -- artificially constructed -- environments. On the other hand, they have demonstrated the ability of experimentally accessible PQCs to solve OpenAI Gym benchmarking tasks. However, it remains an open question whether these near-term QRL techniques can be successfully applied to more complex problems exhibiting high-dimensional observation spaces. In this work, we bridge this gap and present a hybrid model combining a PQC with classical feature encoding and post-processing layers that is capable of tackling Atari games. A classical model, subjected to architectural restrictions similar to those present in the hybrid model is constructed to serve as a reference. Our numerical investigation demonstrates that the proposed hybrid model is capable of solving the Pong environment and achieving scores comparable to the classical reference in Breakout. Furthermore, our findings shed light on important hyperparameter settings and design choices that impact the interplay of the quantum and classical components. This work contributes to the understanding of near-term quantum learning models and makes an important step towards their deployment in real-world RL scenarios.