Discrete LQR and ILQR methods based on high order Runge-Kutta methods

TL;DR

This paper introduces high-order Runge-Kutta based discrete LQR and ILQR methods for optimal control, addressing order reduction issues and demonstrating high efficiency and linear convergence.

Contribution

It reconstructs the equivalence between direct RK discretization and symplectic methods, proposes order conditions for internal controls, and calculates node controls to improve accuracy.

Findings

Third and fourth order explicit RK discretizations face order reduction.

Node control calculation improves control accuracy.

ILQR exhibits linear convergence rate.

Abstract

In this paper, discrete linear quadratic regulator (DLQR) and iterative linear quadratic regulator (ILQR) methods based on high-order Runge-Kutta (RK) discretization are proposed for solving linear and nonlinear quadratic optimal control problems respectively. As discovered in [W. Hager, Runge-Kutta method in optimal control and the discrete adjoint system, Numer. Math.,2000, pp. 247-282], direct approach with RK discretization is equivalent with indirect approach based on symplectic partitioned Runge-Kutta (SPRK) integration. In this paper, we will reconstruct this equivalence by the analogue of continuous and discrete dynamic programming. Then, based on the equivalence, we discuss the issue that the internal-stage controls produced by direct approach may have lower order accuracy than the RK method used. We propose order conditions for internal-stage controls and then demonstrate that third or fourth order explicit RK discretization cannot avoid the order reduction phenomenon. To overcome this obstacle, we calculate node control instead of internal-stage controls in DLQR and ILQR methods. And numerical examples will illustrate the validity of our methods. Another advantage of our methods is high computational efficiency which comes from the usage of feedback technique. In this paper, we also demonstrate that ILQR is essentially a quasi-Newton method with linear convergence rate.