Quantum policy gradient algorithms

TL;DR

This paper develops quantum algorithms for reinforcement learning that can achieve quadratic speed-ups in training policies, especially when using parametrized quantum circuits, highlighting the potential of fully-quantum reinforcement learning.

Contribution

It introduces quantum algorithms for training reinforcement learning policies that exploit quantum interactions, demonstrating advantages with parametrized quantum circuits under certain conditions.

Findings

Quantum algorithms can achieve quadratic speed-ups in sample complexity.

Parametrized quantum circuits produce well-behaved policies for quantum reinforcement learning.

Full quantum reinforcement learning frameworks show promising benefits.

Abstract

Understanding the power and limitations of quantum access to data in machine learning tasks is primordial to assess the potential of quantum computing in artificial intelligence. Previous works have already shown that speed-ups in learning are possible when given quantum access to reinforcement learning environments. Yet, the applicability of quantum algorithms in this setting remains very limited, notably in environments with large state and action spaces. In this work, we design quantum algorithms to train state-of-the-art reinforcement learning policies by exploiting quantum interactions with an environment. However, these algorithms only offer full quadratic speed-ups in sample complexity over their classical analogs when the trained policies satisfy some regularity conditions. Interestingly, we find that reinforcement learning policies derived from parametrized quantum circuits are well-behaved with respect to these conditions, which showcases the benefit of a fully-quantum reinforcement learning framework.