Temporal Parallelisation of the HJB Equation and Continuous-Time Linear Quadratic Control

TL;DR

This paper introduces a method for parallelizing the solution of the Hamilton-Jacobi-Bellman equation in continuous-time optimal control, enabling faster computation by dividing the problem into sub-intervals and solving them concurrently.

Contribution

It develops a novel parallel approach for solving the HJB equation in continuous-time control problems using associative operators and conditional value functions, with closed-form solutions for linear quadratic cases.

Findings

Achieves logarithmic time complexity in solving the control problem.

Demonstrates computational speedup on multi-core CPU and GPU.

Provides closed-form solutions for linear quadratic control problems.

Abstract

This paper presents a mathematical formulation to perform temporal parallelisation of continuous-time optimal control problems, which can be solved via the Hamilton--Jacobi--Bellman (HJB) equation. We divide the time interval of the control problem into sub-intervals, and define a control problem in each sub-interval, conditioned on the start and end states, leading to conditional value functions for the sub-intervals. By defining an associative operator as the minimisation of the sum of conditional value functions, we obtain the elements and associative operators for a parallel associative scan operation. This allows for solving the optimal control problem on the whole time interval in parallel in logarithmic time complexity in the number of sub-intervals. We derive the HJB-type of backward and forward equations for the conditional value functions and solve them in closed form for linear quadratic problems. We also discuss numerical methods for computing the conditional value functions. The computational advantages of the proposed parallel methods are demonstrated via simulations run on a multi-core central processing unit and a graphics processing unit.