Solving high-dimensional Hamilton-Jacobi-Bellman equation with functional hierarchical tensor

TL;DR

This paper introduces a new numerical method using functional hierarchical tensor networks to efficiently solve high-dimensional Hamilton-Jacobi-Bellman equations in stochastic control, overcoming challenges posed by diffusion.

Contribution

It presents a novel regression-based tensor network approach with sketching subroutines for high-dimensional stochastic control problems, including diffusion effects.

Findings

Successfully applied to 1D and 2D Ginzburg-Landau control problems

Achieved accelerated convergence with sketching-based tensor approximations

Demonstrated effectiveness in high-dimensional settings with 64 variables

Abstract

This work proposes a novel numerical scheme for solving the high-dimensional Hamilton-Jacobi-Bellman equation with a functional hierarchical tensor ansatz. We consider the setting of stochastic control, whereby one applies control to a particle under Brownian motion. In particular, the existence of diffusion presents a new challenge to conventional tensor network methods for deterministic optimal control. To overcome the difficulty, we use a general regression-based formulation where the loss term is the Bellman consistency error combined with a Sobolev-type penalization term. We propose two novel sketching-based subroutines for obtaining the tensor-network approximation to the action-value functions and the value functions, which greatly accelerate the convergence for the subsequent regression phase. We apply the proposed approach successfully to two challenging control problems with Ginzburg-Landau potential in 1D and 2D with 64 variables.