Temporal Parallelisation of Dynamic Programming and Linear Quadratic Control

TL;DR

This paper introduces a general framework for parallelizing dynamic programming in optimal control, significantly reducing computation time from linear to logarithmic using parallel scans, with applications to various control problems.

Contribution

It develops a novel formulation enabling parallel computation of dynamic programming, applicable to finite, linear quadratic, and certain nonlinear control problems, demonstrating substantial computational speedups.

Findings

Parallel algorithms achieve logarithmic complexity.

Numerical simulations confirm efficiency on GPU.

Applicable to diverse control problem classes.

Abstract

This paper proposes a general formulation for temporal parallelisation of dynamic programming for optimal control problems. We derive the elements and associative operators to be able to use parallel scans to solve these problems with logarithmic time complexity rather than linear time complexity. We apply this methodology to problems with finite state and control spaces, linear quadratic tracking control problems, and to a class of nonlinear control problems. The computational benefits of the parallel methods are demonstrated via numerical simulations run on a graphics processing unit.