Parameterized Differential Dynamic Programming

TL;DR

This paper introduces Parameterized Differential Dynamic Programming (PDDP), a generalized and convergent algorithm for optimal control with parameters, demonstrated on robotics and urban air mobility systems.

Contribution

It develops a parametric version of DDP with convergence guarantees and applies it to complex systems for control and estimation tasks.

Findings

PDDP converges to a minimum regardless of initialization.

Effective in solving MPC and MHE tasks simultaneously.

Determines optimal transition points in multi-phase flight systems.

Abstract

Differential Dynamic Programming (DDP) is an efficient trajectory optimization algorithm relying on second-order approximations of a system's dynamics and cost function, and has recently been applied to optimize systems with time-invariant parameters. Prior works include system parameter estimation and identifying the optimal switching time between modes of hybrid dynamical systems. This paper generalizes previous work by proposing a general parameterized optimal control objective and deriving a parametric version of DDP, titled Parameterized Differential Dynamic Programming (PDDP). A rigorous convergence analysis of the algorithm is provided, and PDDP is shown to converge to a minimum of the cost regardless of initialization. The effects of varying the optimization to more effectively escape local minima are analyzed. Experiments are presented applying PDDP on multiple robotics systems to solve model predictive control (MPC) and moving horizon estimation (MHE) tasks simultaneously. Finally, PDDP is used to determine the optimal transition point between flight regimes of a complex urban air mobility (UAM) class vehicle exhibiting multiple phases of flight.