Limit-sure reachability for small memory policies in POMDPs is NP-complete

TL;DR

This paper investigates the computational complexity of ensuring near-certain reachability in POMDPs with limited memory, establishing that the problem is NP-complete, which clarifies its difficulty in practical scenarios.

Contribution

It proves that the limit-sure reachability problem for POMDPs with small memory policies is NP-complete, providing new insights into its computational complexity.

Findings

The problem is NP-complete for fixed small memory policies.

Undecidability in general cases contrasts with NP-completeness in bounded memory.

Practical relevance for decision-making in uncertain environments.

Abstract

A standard model that arises in several applications in sequential decision making is partially observable Markov decision processes (POMDPs) where a decision-making agent interacts with an uncertain environment. A basic objective in such POMDPs is the reachability objective, where given a target set of states, the goal is to eventually arrive at one of them. The limit-sure problem asks whether reachability can be ensured with probability arbitrarily close to 1. In general, the limit-sure reachability problem for POMDPs is undecidable. However, in many practical cases the most relevant question is the existence of policies with a small amount of memory. In this work, we study the limit-sure reachability problem for POMDPs with a fixed amount of memory. We establish that the computational complexity of the problem is NP-complete.