Approximation of Singular-Stopping Control Driven by Hawkes Processes via Rescaled MDPs

TL;DR

This paper develops a method to approximate a complex continuous-time singular-stopping control problem driven by Hawkes processes using rescaled Markov Decision Processes, demonstrating convergence and asymptotic optimality.

Contribution

It introduces a rescaling approach linking discrete-time MDPs to continuous-time Hawkes-driven control problems, with proofs of convergence and practical applications.

Findings

Discrete-time value functions converge to continuous-time solutions.

Asymptotic optimality of discrete-time policies is established.

Numerical simulations validate theoretical results.

Abstract

We investigate a singular-optimal stopping stochastic control problem driven by self-exciting dynamics governed by a Hawkes process. In the continuous-time setting, we show that the optimization problem reduces to solving a variational partial differential equation with gradient constraints. We then introduce its discrete-time counterpart, modeled as a Markov Decision Process. We prove that, under an appropriate rescaling procedure, the value function of the discrete-time problem converges to its continuous-time equivalent, implying that the discrete-time optimizers are asymptotically optimal for the continuous-time problem. Finally, we apply these results to an Ornstein-Uhlenbeck stochastic differential equation driven by a Hawkes process with singular control, motivated by optimal power plant investment under cyber threat and we illustrate the theoretical findings through numerical simulations.