Imprecise Probabilities Meet Partial Observability: Game Semantics for Robust POMDPs

TL;DR

This paper advances the theoretical understanding of robust POMDPs by exploring how different uncertainty assumptions influence policies, establishing a game-theoretic semantic framework, and connecting to existing literature.

Contribution

It introduces a game semantics for RPOMDPs, analyzes the impact of uncertainty assumptions on policies, and clarifies the relationship with existing work.

Findings

Different uncertainty assumptions affect optimal policies and values.

RPOMDPs have a POSG semantic with a Nash equilibrium.

Existing RPOMDP literature can be classified using the new semantics.

Abstract

Partially observable Markov decision processes (POMDPs) rely on the key assumption that probability distributions are precisely known. Robust POMDPs (RPOMDPs) alleviate this concern by defining imprecise probabilities, referred to as uncertainty sets. While robust MDPs have been studied extensively, work on RPOMDPs is limited and primarily focuses on algorithmic solution methods. We expand the theoretical understanding of RPOMDPs by showing that 1) different assumptions on the uncertainty sets affect optimal policies and values; 2) RPOMDPs have a partially observable stochastic game (POSG) semantic; and 3) the same RPOMDP with different assumptions leads to semantically different POSGs and, thus, different policies and values. These novel semantics for RPOMDPs give access to results for POSGs, studied in game theory; concretely, we show the existence of a Nash equilibrium. Finally, we classify the existing RPOMDP literature using our semantics, clarifying under which uncertainty assumptions these existing works operate.