# Markov Decision Processes under Ambiguity

## TL;DR

This paper studies Markov Decision Processes with model ambiguity, incorporating risk aversion through entropic risk or AVaR, and provides methods for solving and computing optimal policies.

## Contribution

It introduces a framework for MDPs under ambiguity with risk aversion, proving existence of optimal policies and offering solution techniques.

## Key findings

- Existence of deterministic optimal policies under certain conditions
- Solution methods based on a minimax theorem
- Application to a statistical decision theory example

## Abstract

We consider statistical Markov Decision Processes where the decision maker is risk averse against model ambiguity. The latter is given by an unknown parameter which influences the transition law and the cost functions. Risk aversion is either measured by the entropic risk measure or by the Average Value at Risk. We show how to solve these kind of problems using a general minimax theorem. Under some continuity and compactness assumptions we prove the existence of an optimal (deterministic) policy and discuss its computation. We illustrate our results using an example from statistical decision theory.