On Pioneering Works of Albert Shiryaev on Markov Decision Processes and Some Later Developments

TL;DR

This paper reviews Albert Shiryaev's pioneering contributions to Markov Decision Processes, highlighting key ideas, subsequent developments, and recent research on jump Markov processes and control theory.

Contribution

It provides a comprehensive survey of Shiryaev's fundamental works and explores recent advancements in related stochastic control and jump process research.

Findings

Analysis of Shiryaev's original papers and their impact

Summary of subsequent developments in Markov decision processes

Discussion of recent studies on jump Markov processes

Abstract

This article is dedicated to three fundamental papers on Markov Decision Processes and on control with incomplete observations published by Albert Shiryaev approximately sixty years ago. One of these papers was coauthored with O.V. Viskov. We discuss some of the results and some of many rich ideas presented in these papers and survey some later developments. At the end we mention some recent studies of Albert Shiryaev on Kolmogorov's equations for jump Markov processes and on control of continuous-time jump Markov processes.