Online Planning in POMDPs with State-Requests

TL;DR

This paper introduces AEMS-SR, an online planning algorithm for POMDPs with costly state requests, which efficiently manages the search space and guarantees near-optimal solutions, outperforming existing methods.

Contribution

The paper presents AEMS-SR, a novel graph-based online planning algorithm for POMDPs with state requests, providing theoretical guarantees and empirical improvements over state-of-the-art algorithms.

Findings

AEMS-SR is $oldsymbol{ extit{ ext{ε}}}$-optimal.

AEMS-SR outperforms AEMS and POMCP in empirical tests.

Efficient planning in domains with costly state requests.

Abstract

In key real-world problems, full state information is sometimes available but only at a high cost, like activating precise yet energy-intensive sensors or consulting humans, thereby compelling the agent to operate under partial observability. For this scenario, we propose AEMS-SR (Anytime Error Minimization Search with State Requests), a principled online planning algorithm tailored for POMDPs with state requests. By representing the search space as a graph instead of a tree, AEMS-SR avoids the exponential growth of the search space originating from state requests. Theoretical analysis demonstrates AEMS-SR's $\varepsilon$-optimality, ensuring solution quality, while empirical evaluations illustrate its effectiveness compared with AEMS and POMCP, two SOTA online planning algorithms. AEMS-SR enables efficient planning in domains characterized by partial observability and costly state requests offering practical benefits across various applications.