Quantum speedups for convex dynamic programming

TL;DR

This paper introduces a quantum algorithm for solving convex dynamic programming problems, providing significant speedups over classical methods, especially for high-dimensional state spaces and certain problem classes.

Contribution

The paper presents a novel quantum algorithm that efficiently solves convex dynamic programming problems, including hard stochastic cases, with provable speedups over classical algorithms.

Findings

Quantum algorithm solves convex DP with exponential speedup in state space size.

Achieves quadratic speedup compared to classical Bellman methods for some problems.

Applicable to hard stochastic dynamic programming problems.

Abstract

We present a quantum algorithm to solve dynamic programming problems with convex value functions. For linear discrete-time systems with a $d$-dimensional state space of size $N$, the proposed algorithm outputs a quantum-mechanical representation of the value function in time $O(T \gamma^{dT}\mathrm{polylog}(N,(T/\varepsilon)^{d}))$, where $\varepsilon$ is the accuracy of the solution, $T$ is the time horizon, and $\gamma$ is a problem-specific parameter depending on the condition numbers of the cost functions. This allows us to evaluate the value function at any fixed state in time $O(T \gamma^{dT}\sqrt{N}\,\mathrm{polylog}(N,(T/\varepsilon)^{d}))$, and the corresponding optimal action can be recovered by solving a convex program. The class of optimization problems to which our algorithm can be applied includes provably hard stochastic dynamic programs. Finally, we show that the algorithm obtains a quadratic speedup (up to polylogarithmic factors) compared to the classical Bellman approach on some dynamic programs with continuous state space that have $\gamma=1$.