Long run control of nonhomogeneous Markov processes

TL;DR

This paper investigates the long-term behavior of controlled nonhomogeneous Markov processes, establishing existence, continuity, and stability of solutions to associated Bellman equations for average reward and risk-sensitive functionals.

Contribution

It introduces new results on the existence, continuity, and stability of solutions to Bellman equations in the context of nonhomogeneous Markov processes.

Findings

Existence of solutions to Bellman equations for average reward and risk-sensitive functionals.

Continuity of value functions with respect to risk parameters.

Stability of functionals under pointwise convergence of controls.

Abstract

In the paper average reward per unit time and average risk sensitive reward functionals are considered for controlled nonhomogeneous Markov processes. Existence of solutions to suitable Bellman equations is shown. Continuity of the value functions with respect to risk parameter is also proved. Finally stability of functionals with respect to pointwise convergence of Markov controls is studied.