Optimal Impulse Control of Dynamical Systems

TL;DR

This paper develops a dynamic programming framework for optimal impulse control of deterministic systems using Markov Decision Processes, establishing theoretical equivalences and illustrating with epidemiological examples.

Contribution

It introduces a rigorous approach to impulse control in deterministic systems via MDPs, proving equivalence of integral and differential optimality equations.

Findings

Established the dynamic programming approach for impulse control.

Proved the equivalence of integral and differential forms of the optimality equation.

Demonstrated applicability with an epidemiological example.

Abstract

Using the tools of the Markov Decision Processes, we justify the dynamic programming approach to the optimal impulse control of deterministic dynamical systems. We prove the equivalence of the integral and differential forms of the optimality equation. The theory is illustrated by an example from mathematical epidemiology. The developed methods can be also useful for the study of piecewise deterministic Markov processes.