On the Computational Complexity of Stochastic Controller Optimization in POMDPs

TL;DR

This paper proves that optimizing stochastic controllers in POMDPs is NP-hard, highlighting the computational difficulty, but also identifies a special convex case with efficient solutions.

Contribution

It establishes the NP-hardness of stochastic controller optimization in POMDPs and presents a special convex case with tractable solutions.

Findings

Optimal stochastic 'blind' controller problem is NP-hard.

Decision problem is NP-hard, in PSPACE, and SQRT-SUM-hard.

A special convex case admits efficient global solutions.

Abstract

We show that the problem of finding an optimal stochastic 'blind' controller in a Markov decision process is an NP-hard problem. The corresponding decision problem is NP-hard, in PSPACE, and SQRT-SUM-hard, hence placing it in NP would imply breakthroughs in long-standing open problems in computer science. Our result establishes that the more general problem of stochastic controller optimization in POMDPs is also NP-hard. Nonetheless, we outline a special case that is convex and admits efficient global solutions.