A pseudo-Markov property for controlled diffusion processes

TL;DR

This paper provides two rigorous methods to justify the pseudo-Markov property in controlled diffusion processes, which is crucial for establishing the dynamic programming principle in stochastic control.

Contribution

It introduces two approaches—one based on Fleming and Souganidis' sketch and another on state space enlargement—to rigorously justify the pseudo-Markov property.

Findings

Two rigorous approaches to pseudo-Markov property

Clarification of measurability and topological issues

Enhanced understanding of dynamic programming in stochastic control

Abstract

In this note, we propose two different approaches to rigorously justify a pseudo-Markov property for controlled diffusion processes which is often (explicitly or implicitly) used to prove the dynamic programming principle in the stochastic control literature. The first approach develops a sketch of proof proposed by Fleming and Souganidis~\cite{fleming-souganidis}. The second approach is based on an enlargement of the original state space and a controlled martingale problem. We clarify some measurability and topological issues raised by these two approaches.