Quantum POMDPs

TL;DR

This paper introduces quantum observable Markov decision processes (QOMDPs), extending classical POMDPs into the quantum domain, and analyzes their computational complexity and decidability properties.

Contribution

It defines QOMDPs as quantum analogues of POMDPs and compares their policy existence problems, revealing undecidability in certain quantum scenarios.

Findings

Policy existence complexity matches classical POMDPs in some cases

Reaching goal states is decidable in POMDPs but undecidable in QOMDPs

Quantum extension introduces new computational challenges

Abstract

We present quantum observable Markov decision processes (QOMDPs), the quantum analogues of partially observable Markov decision processes (POMDPs). In a QOMDP, an agent's state is represented as a quantum state and the agent can choose a superoperator to apply. This is similar to the POMDP belief state, which is a probability distribution over world states and evolves via a stochastic matrix. We show that the existence of a policy of at least a certain value has the same complexity for QOMDPs and POMDPs in the polynomial and infinite horizon cases. However, we also prove that the existence of a policy that can reach a goal state is decidable for goal POMDPs and undecidable for goal QOMDPs.