# A Class of Solvable Markov Decision Models with Incomplete Information

## TL;DR

This paper establishes conditions under which optimal policies exist for a broad class of Markov decision models with incomplete information, extending known results for POMDPs by introducing semi-uniform Feller transition probabilities.

## Contribution

It introduces the semi-uniform Feller transition probability concept and proves its equivalence between MDPII and belief MDPs, broadening the understanding of optimal policy existence.

## Key findings

- Semi-uniform Feller transition probability is key for optimal policies.
- Equivalence of transition probabilities between MDPII and belief MDPs.
- Extends known conditions for POMDPs to more general models.

## Abstract

This paper investigates natural conditions for the existence of optimal policies for a Markov decision process with incomplete information (MDPII) and with expected total costs. The MDPII is the classic model of a controlled stochastic process with incomplete state observations which is more general than Partially Observable Markov Decision Processes (POMDPs). For MDPIIs we introduce the notion of a semi-uniform Feller transition probability, which is stronger than the notion of a weakly continuous transition probability. We show that an MDPII has a semi-uniform Feller transition probability if and only if the corresponding belief MDP also has a semi-uniform Feller transition probability. This fact has several corollaries. In particular, it provides new and implies all known sufficient conditions for the existence of optimal policies for POMDPs with expected total costs