Quantum-enhanced policy iteration on the example of a mountain car

TL;DR

This paper introduces a quantum-enhanced policy iteration algorithm tailored for reinforcement learning, specifically validated on the mountain car problem, demonstrating potential speedups with quantum computing techniques.

Contribution

It proposes a novel quantum-enhanced policy iteration method and analyzes its complexity and performance in continuous state reinforcement learning tasks.

Findings

Numerical results verify the QEPI algorithm on a mountain car example.

Benchmarking shows QEPI's potential advantages over classical methods.

The paper discusses stochastic discretization for continuous state problems.

Abstract

Advances in the experimental demonstration of quantum processors have provoked a surge of interest to the idea of practical implementation of quantum computing over last years. It is expected that the use of quantum algorithms will significantly speed up the solution to certain problems in numerical optimization and machine learning. In this paper, we propose a quantum-enhanced policy iteration (QEPI) algorithm as widely used in the domain of reinforcement learning and validate it with the focus on the mountain car problem. In practice, we elaborate on the soft version of the value iteration algorithm, which is beneficial for policy interpretation, and discuss the stochastic discretization technique in the context of continuous state reinforcement learning problems for the purposes of QEPI. The complexity of the algorithm is analyzed for dense and (typical) sparse cases. Numerical results on the example of a mountain car with the use of a quantum emulator verify the developed procedures and benchmark the QEPI performance.