On the expected total reward with unbounded returns for Markov decision processes

TL;DR

This paper investigates optimal strategies in Markov decision processes with unbounded rewards, establishing existence results under broad conditions and introducing a new approach for weak convergence of probability measures.

Contribution

It provides the first general existence theorem for optimal strategies in MDPs with unbounded rewards and non-compact action sets, using a novel weak convergence characterization.

Findings

Existence of optimal strategies under broad conditions

New characterization for weak convergence of probability measures

Illustrative examples demonstrating theoretical results

Abstract

We consider a discrete-time Markov decision process with Borel state and action spaces. The performance criterion is to maximize a total expected {utility determined by unbounded return function. It is shown the existence of optimal strategies under general conditions allowing the reward function to be unbounded both from above and below and the action sets available at each step to the decision maker to be not necessarily compact. To deal with unbounded reward functions, a new characterization for the weak convergence of probability measures is derived. Our results are illustrated by examples.