A Possibilistic Model for Qualitative Sequential Decision Problems under Uncertainty in Partially Observable Environments

TL;DR

This paper introduces a possibilistic, qualitative model for sequential decision problems in partially observable environments, offering a finite belief state space alternative to traditional stochastic POMDPs.

Contribution

It develops a possibilistic counterpart to POMDPs using ordinal uncertainty, enabling finite belief spaces even with large or exponential state spaces.

Findings

Belief state space remains finite in the possibilistic model.

The model effectively handles uncertainty without stochastic assumptions.

It offers a computational advantage over stochastic POMDPs.

Abstract

In this article we propose a qualitative (ordinal) counterpart for the Partially Observable Markov Decision Processes model (POMDP) in which the uncertainty, as well as the preferences of the agent, are modeled by possibility distributions. This qualitative counterpart of the POMDP model relies on a possibilistic theory of decision under uncertainty, recently developed. One advantage of such a qualitative framework is its ability to escape from the classical obstacle of stochastic POMDPs, in which even with a finite state space, the obtained belief state space of the POMDP is infinite. Instead, in the possibilistic framework even if exponentially larger than the state space, the belief state space remains finite.