Average Cost Markov Decision Processes with Semi-Uniform Feller Transition Probabilities

TL;DR

This paper investigates average-cost Markov decision processes with semi-uniform Feller transition probabilities, focusing on optimality conditions, policy existence, and approximation methods for incomplete information scenarios.

Contribution

It establishes the validity of optimality inequalities, proves the existence of optimal policies, and explores approximation techniques within this specific class of MDPs.

Findings

Optimality inequalities are valid for the class.

Optimal policies exist under certain conditions.

Approximation of average-cost policies by discounted-cost policies is feasible.

Abstract

This paper studies average-cost Markov decision processes with semi-uniform Feller transition probabilities. This class of MDPs was recently introduced by the authors to study MDPs with incomplete information. This paper studies the validity of optimality inequalities, the existence of optimal policies, and the approximations of optimal policies by policies optimizing total discounted costs.