Benchmarking Quantum Reinforcement Learning

TL;DR

This paper provides a comprehensive benchmark comparison of major quantum reinforcement learning algorithms using gridworld games, aiming to evaluate their performance and the role of quantum principles.

Contribution

It introduces a unified benchmarking framework and metrics to evaluate different QRL classes, clarifying their strengths, weaknesses, and potential advantages over classical RL.

Findings

PQC-QRL outperforms others in certain gridworld tasks.

Quantum principles' impact on RL performance remains inconclusive.

Benchmarking reveals specific scenarios where quantum approaches excel.

Abstract

Quantum Reinforcement Learning (QRL) has emerged as a promising research field, leveraging the principles of quantum mechanics to enhance the performance of reinforcement learning (RL) algorithms. However, despite its growing interest, QRL still faces significant challenges. It is still uncertain if QRL can show any advantage over classical RL beyond artificial problem formulations. Additionally, it is not yet clear which streams of QRL research show the greatest potential. The lack of a unified benchmark and the need to evaluate the reliance on quantum principles of QRL approaches are pressing questions. This work aims to address these challenges by providing a comprehensive comparison of three major QRL classes: Parameterized Quantum Circuit based QRL (PQC-QRL) (with one policy gradient (QPG) and one Q-Learning (QDQN) algorithm), Free Energy based QRL (FE-QRL), and Amplitude Amplification based QRL (AA-QRL). We introduce a set of metrics to evaluate the QRL algorithms on the widely applicable benchmark of gridworld games. Our results provide a detailed analysis of the strengths and weaknesses of the QRL classes, shedding light on the role of quantum principles in QRL and paving the way for future research in this field.