Kolmogorov's Equations for Jump Markov Processes and their Applications to Control Problems

TL;DR

This paper explores the solutions to Kolmogorov's equations for jump Markov processes, including explosive cases, and discusses their applications to controlling jump stochastic systems, extending classical results to more general processes.

Contribution

It provides a comprehensive survey of solutions to Kolmogorov's equations for both nonexplosive and explosive jump Markov processes and applies these to control problems.

Findings

Extended Kolmogorov equations to explosive processes

Analyzed solutions for nonhomogeneous jump Markov processes

Applied results to control of jump stochastic systems

Abstract

This paper describes the structure of solutions to Kolmogorov's equations for nonhomogeneous jump Markov processes and applications of these results to control of jump stochastic systems. These equations were studied by Feller (1940), who clarified in 1945 in the errata to that paper that some of its results covered only nonexplosive Markov processes. In this work, which is largely of a survey nature, the case of explosive processes is also considered. This paper is based on the invited talk presented by the authors at the conference "Chebyshev-200", and it describes the results of their joined studies with Manasa Mandava (1984-2019).