Average Continuous Control of Piecewise Deterministic Markov Processes

TL;DR

This paper addresses the long-term average control problem for piecewise deterministic Markov processes, deriving optimality equations, establishing the existence of feedback controls, and providing conditions for optimal solutions in a general Borel space setting.

Contribution

It introduces a novel approach linking the average cost optimality equation to an integro-differential equation for PDMPs, and proves the existence of optimal feedback controls.

Findings

Derived an optimality equation for long run average cost.

Proved the existence of a measurable feedback selector.

Established conditions for the existence of optimal controls.

Abstract

This paper deals with the long run average continuous control problem of piecewise deterministic Markov processes (PDMP's) taking values in a general Borel space and with compact action space depending on the state variable. The control variable acts on the jump rate and transition measure of the PDMP, and the running and boundary costs are assumed to be positive but not necessarily bounded. Our first main result is to obtain an optimality equation for the long run average cost in terms of a discrete-time optimality equation related to the embedded Markov chain given by the post-jump location of the PDMP. Our second main result guarantees the existence of a feedback measurable selector for the discrete-time optimality equation by establishing a connection between this equation and an integro-differential equation. Our final main result is to obtain some sufficient conditions for the existence of a solution for a discrete-time optimality inequality and an ordinary optimal feedback control for the long run average cost using the so-called vanishing discount approach.