An Actor-Critic Framework for Continuous-Time Jump-Diffusion Controls with Normalizing Flows

TL;DR

This paper introduces an actor-critic method utilizing normalizing flows for continuous-time jump-diffusion control problems, effectively handling high-dimensional, time-dependent stochastic dynamics with jumps.

Contribution

It develops a mesh-free, policy-gradient framework with expressive stochastic policies using conditional normalizing flows, suitable for complex jump-diffusion control and game problems.

Findings

Stable learning despite jump discontinuities

Accurate approximation of optimal policies

Favorable scaling with dimension and agents

Abstract

Continuous-time stochastic control with time-inhomogeneous jump-diffusion dynamics is central in finance and economics, but computing optimal policies is difficult under explicit time dependence, discontinuous shocks, and high dimensionality. We propose an actor-critic framework that serves as a mesh-free solver for entropy-regularized control problems and stochastic games with jumps. The approach is built on a time-inhomogeneous little q-function and an appropriate occupation measure, yielding a policy-gradient representation that accommodates time-dependent drift, volatility, and jump terms. To represent expressive stochastic policies in continuous-action spaces, we parameterize the actor using conditional normalizing flows, enabling flexible non-Gaussian policies while retaining exact likelihood evaluation for entropy regularization and policy optimization. We validate the method on time-inhomogeneous linear-quadratic control, Merton portfolio optimization, and a multi-agent portfolio game, using explicit solutions or high-accuracy benchmarks. Numerical results demonstrate stable learning under jump discontinuities, accurate approximation of optimal stochastic policies, and favorable scaling with respect to dimension and number of agents.