Markov control of continuous time Markov processes with long run functionals by time discretization

TL;DR

This paper explores how to control continuous time Markov processes by approximating them with discrete time models, focusing on long-term rewards and stability of control strategies.

Contribution

It introduces a method to analyze continuous time Markov control problems through time discretization, addressing long run functionals and stability issues.

Findings

Discretization approach effectively approximates continuous time control.

Long run functionals are stable under pointwise convergence of controls.

Provides insights into risk-sensitive and average reward criteria.

Abstract

In the paper we study continuous time controlled Markov processes using discrete time controlled Markov processes. We consider long run functionals: average reward per unit time or long run risk sensitive functional. We also investigate stability of continuous time functionals with respect to pointwise convergence of Markov controls.