Sensor Synthesis for POMDPs with Reachability Objectives

TL;DR

This paper introduces a symbolic SAT-based method for synthesizing minimal additional sensors in POMDPs to ensure almost-sure reachability with small-memory policies, reducing sensor complexity without increasing policy complexity.

Contribution

It presents the first approach to synthesize the weakest sensors in POMDPs for reachability objectives, with a formal NP-complete analysis and practical implementation.

Findings

Significant reduction in the number of sensors needed in classical POMDP examples.

The approach maintains policy complexity while decreasing sensor requirements.

The problem is shown to be NP-complete, guiding the design of the algorithm.

Abstract

Partially observable Markov decision processes (POMDPs) are widely used in probabilistic planning problems in which an agent interacts with an environment using noisy and imprecise sensors. We study a setting in which the sensors are only partially defined and the goal is to synthesize "weakest" additional sensors, such that in the resulting POMDP, there is a small-memory policy for the agent that almost-surely (with probability~1) satisfies a reachability objective. We show that the problem is NP-complete, and present a symbolic algorithm by encoding the problem into SAT instances. We illustrate trade-offs between the amount of memory of the policy and the number of additional sensors on a simple example. We have implemented our approach and consider three classical POMDP examples from the literature, and show that in all the examples the number of sensors can be significantly decreased (as compared to the existing solutions in the literature) without increasing the complexity of the policies.