Qualitative Analysis of Partially-observable Markov Decision Processes

TL;DR

This paper investigates the computational complexity and memory requirements for observation-based strategies in POMDPs with omega-regular objectives, providing new bounds and a complete complexity classification.

Contribution

It offers a comprehensive complexity analysis and optimal bounds on memory for strategies in POMDPs with parity objectives, advancing understanding of qualitative analysis.

Findings

Established new upper and lower bounds for complexity.

Provided optimal bounds on memory requirements.

Completed a full complexity classification for POMDPs with parity objectives.

Abstract

We study observation-based strategies for partially-observable Markov decision processes (POMDPs) with omega-regular objectives. An observation-based strategy relies on partial information about the history of a play, namely, on the past sequence of observations. We consider the qualitative analysis problem: given a POMDP with an omega-regular objective, whether there is an observation-based strategy to achieve the objective with probability~1 (almost-sure winning), or with positive probability (positive winning). Our main results are twofold. First, we present a complete picture of the computational complexity of the qualitative analysis of POMDP s with parity objectives (a canonical form to express omega-regular objectives) and its subclasses. Our contribution consists in establishing several upper and lower bounds that were not known in literature. Second, we present optimal bounds (matching upper and lower bounds) on the memory required by pure and randomized observation-based strategies for the qualitative analysis of POMDP s with parity objectives and its subclasses.