Open-loop Deterministic Density Control of Marked Jump Diffusions

TL;DR

This paper develops an open-loop, deterministic optimal control method for large populations of agents modeled by marked jump diffusion equations, using a sampling-based algorithm to handle complex nonlinear dynamics.

Contribution

It introduces a novel open-loop control synthesis framework for density control of jump diffusion processes, linking IDMP with dynamic programming and providing a practical sampling-based solution.

Findings

Established the relationship between IDMP and dynamic programming.

Developed a sampling-based control algorithm for complex nonlinear dynamics.

Demonstrated the approach on agents with non-affine drift and noise.

Abstract

The standard practice in modeling dynamics and optimal control of a large population, ensemble, multi-agent system represented by it's continuum density, is to model individual decision making using local feedback information. In comparison to a closed-loop optimal control scheme, an open-loop strategy, in which a centralized controller broadcasts identical control signals to the ensemble of agents, mitigates the computational and infrastructure requirements for such systems. This work considers the open-loop, deterministic and optimal control synthesis for the density control of agents governed by marked jump diffusion stochastic diffusion equations. The density evolves according to a forward-in-time Chapman-Kolmogorov partial integro-differential equation and the necessary optimality conditions are obtained using the infinite dimensional minimum principle (IDMP). We establish the relationship between the IDMP and the dynamic programming principle as well as the IDMP and stochastic dynamic programming for the synthesized controller. Using the linear Feynman-Kac lemma, a sampling-based algorithm to compute the control is presented and demonstrated for agent dynamics with non-affine and nonlinear drift as well as noise terms.