Continuous time Markov chain based approximation of stationary and weak KAM Hamilton-Jacobi equations

TL;DR

This paper introduces a novel approximation method for stationary and weak KAM Hamilton-Jacobi equations using continuous-time Markov decision processes, leading to algebraic systems that serve as numerical schemes.

Contribution

It develops a Markov decision process-based approach to approximate weak KAM Hamilton-Jacobi equations, bridging calculus of variations and stochastic control.

Findings

Convergence of the Markov decision process approximation to the weak KAM solutions.

Derivation of algebraic systems as numerical schemes for the equations.

Establishment of convergence of Mather measures in the approximation.

Abstract

Main objects of the paper are stationary and weak KAM Hamilton-Jacobi equations on the finite-dimensional torus. The key idea of the paper is to replace the underlying calculus of variations problems with continuous time Markov decision problems. This directly leads to an approximation of the stationary Hamilton-Jacobi equation by the Bellman equation for a discounting Markov decision problem. Developing elements of the weak KAM theory for the Markov decision problem, we obtain an approximation of the effective Hamiltonian. Additionally, convergences of the functional parts of the discrete weak KAM equations and Mather measures are shown. It turns out that the approximating equations are systems of algebraic equations. Thus, the paper's result can be seen as numerical schemes for stationary and weak KAM Hamilton-Jacobi equations.