On Global and Local Convergence of Iterative Linear Quadratic Optimization Algorithms for Discrete Time Nonlinear Control

TL;DR

This paper analyzes the convergence properties of iterative linear quadratic optimization algorithms, like ILQR and DDP, for discrete-time nonlinear control, providing guarantees and rates under various conditions.

Contribution

It offers detailed convergence guarantees, local and global, for ILQR and DDP, including conditions for global convergence and complexity bounds.

Findings

Global convergence to minima when dynamics are surjective and costs are gradient dominated.

Quadratic local convergence when costs are self-concordant.

Surjectivity of linearized dynamics holds under certain discretization schemes.

Abstract

A classical approach for solving discrete time nonlinear control on a finite horizon consists in repeatedly minimizing linear quadratic approximations of the original problem around current candidate solutions. While widely popular in many domains, such an approach has mainly been analyzed locally. We provide detailed convergence guarantees to stationary points as well as local linear convergence rates for the Iterative Linear Quadratic Regulator (ILQR) algorithm and its Differential Dynamic Programming (DDP) variant. For problems without costs on control variables, we observe that global convergence to minima can be ensured provided that the linearized discrete time dynamics are surjective, costs on the state variables are gradient dominated. We further detail quadratic local convergence when the costs are self-concordant. We show that surjectivity of the linearized dynamics hold for appropriate discretization schemes given the existence of a feedback linearization scheme. We present complexity bounds of algorithms based on linear quadratic approximations through the lens of generalized Gauss-Newton methods. Our analysis uncovers several convergence phases for regularized generalized Gauss-Newton algorithms.