Adaptive Learning via Off-Model Training and Importance Sampling for Fully Non-Markovian Optimal Stochastic Control. Complete version

TL;DR

This paper introduces a Monte Carlo learning framework for non-Markovian stochastic control problems, enabling off-model training and adaptive updates using importance sampling, with theoretical error bounds and numerical validation.

Contribution

It develops explicit training laws and importance weights for non-Markovian systems, allowing reweighting of a fixed dataset for different models and adaptive recalibration without new trajectory generation.

Findings

Non-asymptotic error bounds for neural network approximation of dynamic programming.

Quantitative estimates separating Monte Carlo error from model risk.

Numerical experiments demonstrating off-model training and adaptive importance sampling.

Abstract

This paper studies continuous-time stochastic control problems whose controlled states are fully non-Markovian and depend on unknown model parameters. Such problems arise naturally in path-dependent stochastic differential equations, rough-volatility hedging, and systems driven by fractional Brownian motion. Building on the discrete skeleton approach developed in earlier work, we propose a Monte Carlo learning methodology for the associated embedded backward dynamic programming equation. Our main contribution is twofold. First, we construct explicit dominating training laws and Radon--Nikodym weights for several representative classes of non-Markovian controlled systems. This yields an off-model training architecture in which a fixed synthetic dataset is generated under a reference law, while the dynamic programming operators associated with a target model are recovered by importance sampling. Second, we use this structure to design an adaptive update mechanism under parametric model uncertainty, so that repeated recalibration can be performed by reweighting the same training sample rather than regenerating new trajectories. For fixed parameters, we establish non-asymptotic error bounds for the approximation of the embedded dynamic programming equation via deep neural networks. For adaptive learning, we derive quantitative estimates that separate Monte Carlo approximation error from model-risk error. Numerical experiments illustrate both the off-model training mechanism and the adaptive importance-sampling update in structured linear-quadratic examples.