Convexity of a stochastic control functional related to importance sampling of It\^o diffusions

TL;DR

This paper investigates the convexity properties of a stochastic control functional related to importance sampling of Itô diffusions, providing conditions for positive definiteness and implications for solving Hamilton-Jacobi-Bellman equations efficiently.

Contribution

It establishes sufficient conditions for the convexity of the control functional, enabling dimension-robust solutions to related HJB boundary value problems.

Findings

Conditions for twice-differentiability and Hölder continuity of the value function.

Expression for the second variation using Kazamaki's criterion.

Finiteness of the moment generating function of the first exit time.

Abstract

We consider the problem of rare event importance sampling, where the random variable of interest is a path functional of an It\^o diffusion computed up to the first exit from a $d$-dimensional bounded domain. Dupuis and Wang (\textit{Ann. Appl. Probab.}, 15 (2005), pp. 1-38) studied the importance sampling problem by formulating it as a stochastic optimal control problem, where the value function is related to the conditional cumulant generating function of the random variable. In this paper, we show that the sufficient conditions for the value function to be twice-differentiable and $\alpha$-uniformly H\"older continuous on the closure of the domain are also sufficient conditions for positive definiteness of the second variation of the control functional on the space of differentiable, $\alpha$-uniformly H\"older continuous $\mathbb{R}^d$-valued feedback controls. We derive an expression for the second variation using Kazamaki's sufficient condition for $L^q$-boundedness of exponential martingales, and using Fredholm theory to prove the finiteness of the moment generating function of the first exit time over any bounded interval containing the origin. The strict convexity result suggests that one may be able to solve the corresponding Hamilton-Jacobi-Bellman boundary value problem in a dimension-robust way, by combining convex optimisation and Monte Carlo methods. We apply the result to analyse a gradient descent algorithm proposed by Hartmann and Sch\"utte (\textit{J. Stat. Mech. Theor. Exp.} (2012), P11004) for efficient rare event simulation.