Dual Approaches to Stochastic Control via SPDEs and the Pathwise Hopf Formula

TL;DR

This paper introduces dual methods for stochastic control problems using SPDEs and the Hopf formula, enabling high-dimensional solutions and providing dual bounds that complement existing primal approaches.

Contribution

It formulates the inner optimization as an SPDE and proves the generalized Hopf formula, extending prior conjectures and enabling curse-of-dimensionality-free solutions.

Findings

Dual approaches effectively compute bounds in high dimensions.

Numerical experiments show complementarity with primal methods.

Proved the generalized Hopf formula under mild conditions.

Abstract

We develop dual approaches for continuous-time stochastic control problems, enabling the computation of robust dual bounds in high-dimensional state and control spaces. Building on the dual formulation proposed in [L. C. G. Rogers, SIAM Journal on Control and Optimization, 46 (2007), pp. 1116--1132], we first formulate the inner optimization problem as a stochastic partial differential equation (SPDE); the expectation of its solution yields the dual bound. Curse-of-dimensionality-free methods are proposed based on the Pontryagin maximum principle and the generalized Hopf formula. In the process, we prove the generalized Hopf formula, first introduced as a conjecture in [Y. T. Chow, J. Darbon, S. Osher, and W. Yin, Journal of Computational Physics 387 (2019), pp. 376--409], under mild conditions. Numerical experiments demonstrate that our dual approaches effectively complement primal methods, including the deep BSDE method for solving high-dimensional PDEs and the deep actor-critic method in reinforcement learning.