The turnpike theorems for Markov games

TL;DR

This paper reviews the development of turnpike theorems and establishes new stochastic turnpike results for finite-horizon two-person zero-sum Markov games on general Borel spaces, focusing on strategies and state distributions.

Contribution

It introduces stochastic versions of the early and middle turnpike theorems for Markov games using Bellman and Shapley operators, extending classical results.

Findings

Proves stochastic early turnpike theorem for optimal strategies.

Establishes stochastic middle turnpike theorem for state distributions.

Extends turnpike theory to general Borel state spaces.

Abstract

This paper has a two-folded purpose. First, we attempt to outline the development of the turnpike theorems in the the last several decades. Second, we study turnpike theorems in finite-horizon two-person zero-sum Markov games on a general Borel state space. Utilising the Bellman (or Shapley) operator defined for this game, we prove the stochastic versions of the early turnpike theorem on the set of optimal strategies and the middle turnpike theorem on the distribution of the state space.