Bounding the Error of Value Functions in Sobolev Norm Yields Bounds on Suboptimality of Controller Performance

TL;DR

This paper links Sobolev norm bounds on approximate solutions of the HJB equation to guarantees on controller suboptimality, showing convergence in the Sobolev $W^{1, abla}$ norm ensures near-optimal control performance.

Contribution

It establishes a novel connection between Sobolev $W^{1, abla}$ norm bounds on value functions and suboptimality guarantees for nonlinear control systems.

Findings

Suboptimality is bounded by the $L^ abla$ norm of the HJB residual.

Convergence in $W^{1, abla}$ norm guarantees controllers near the true optimum.

Weaker norms like $W^{1,p}$ do not provide such guarantees.

Abstract

Optimal feedback controllers for nonlinear systems can be derived by solving the Hamilton-Jacobi-Bellman (HJB) equation. However, because the HJB is a nonlinear partial differential equation, numerical methods typically provide only approximate solutions. While numerical error bounds on approximate HJB solutions are often available, these bounds do not necessarily translate into guarantees on the suboptimality of the resulting controllers. In this paper, we establish that the suboptimality of the resulting controller is bounded by the $L^\infty$ norm of the HJB residual, which is, in turn, bounded by numerical error in the value function as measured in the Sobolev $W^{1,\infty}$ norm. This implies that convergence of value functions in $W^{1,\infty}$ result in controllers that yield a cost that is arbitrarily close to the true minimum. In contrast, we demonstrate that such guarantees do not hold when the value function error is measured in weaker norms, such as the Sobolev $W^{1,p}$ norm for finite $p$. These results apply to systems governed by Lipschitz continuous dynamics over a finite time horizon with compact input space.