# An axiomatic approach to Markov decision processes

## TL;DR

This paper develops an axiomatic framework for finite Markov decision processes with zero discount rate, establishing preference-based conditions for the existence of optimal policies in such settings.

## Contribution

It introduces preference foundations for 0-discount and average overtaking optimality, addressing the challenge of non-existence of optimal policies in no-discount scenarios.

## Key findings

- Preference conditions ensuring optimal policies exist
- Axiomatic foundations for zero-discount MDPs
- Implications for control, games, and economics

## Abstract

This paper presents an axiomatic approach to finite Markov decision processes where the discount rate is zero. One of the principal difficulties in the no discounting case is that, even if attention is restricted to stationary policies, a strong overtaking optimal policy need not exists. We provide preference foundations for two criteria that do admit optimal policies: $0$-discount optimality and average overtaking optimality. As a corollary of our results, we obtain conditions on a decision maker's preferences which ensure that an optimal policy exists. These results have implications for disciplines where stochastic dynamic programming problems arise, including automatic control, dynamic games, and economic development.