Convergence Analysis of the Upwind Difference Methods for Hamilton-Jacobi-Bellman Equations

TL;DR

This paper analyzes the convergence of upwind difference schemes for Hamilton-Jacobi-Bellman equations, demonstrating first-order convergence to classical solutions and convergence of optimal controls, supported by numerical experiments.

Contribution

It provides the first-order convergence proof for classical solutions and establishes control convergence, enhancing understanding of numerical methods for HJB equations.

Findings

Numerical solutions converge at a first-order rate for classical solutions.

Optimal control inputs also converge under the scheme.

Numerical experiments confirm theoretical convergence results.

Abstract

This paper investigates the convergence properties of the upwind difference scheme for the Hamilton--Jacobi--Bellman (HJB) equation, a central partial differential equation in optimal control theory. First, assuming the existence of a classical solution, we show that the numerical solution converges to the true solution with a first-order rate with respect to the time step. This result complements the square-root rate established in previous studies for viscosity solutions. Second, by exploiting the correspondence between HJB equations and conservation laws, we prove the convergence of the optimal control input. This analysis is crucial for practical applications where the control input is the primary quantity of interest, yet it has rarely been addressed in previous studies. Finally, we confirm the validity of our theoretical results through numerical experiments on typical control problems.