Observer-Aware Probabilistic Planning Under Partial Observability

TL;DR

This paper introduces a framework for observer-aware probabilistic planning under partial observability, enabling agents to optimize information transmission and handle dynamic hidden variables in realistic scenarios.

Contribution

It extends observer-aware Markov decision processes to partial observability, formalizing properties like legibility and predictability, and analyzing convergence behavior in benchmark problems.

Findings

PO-OAMDPs can model dynamic hidden variables.

Dedicated initializations improve HSVI convergence.

Strategies effectively balance information transmission and goal achievement.

Abstract

In this article, we are interested in planning problems where the agent is aware of the presence of an observer, and where this observer is in a partial observability situation. The agent has to choose its strategy so as to optimize the information transmitted by observations. Building on observer-aware Markov decision processes (OAMDPs), we propose a framework to handle this type of problems and thus formalize properties such as legibility, explicability and predictability. This extension of OAMDPs to partial observability can not only handle more realistic problems, but also permits considering dynamic hidden variables of interest. These dynamic target variables allow, for instance, working with predictability, or with legibility problems where the goal might change during execution. We discuss theoretical properties of PO-OAMDPs and, experimenting with benchmark problems, we analyze HSVI's convergence behavior with dedicated initializations and study the resulting strategies.