A Unified Perspective on Multiple Shooting In Differential Dynamic Programming

TL;DR

This paper extends Differential Dynamic Programming to multiple shooting, enhancing robustness and convergence through a novel derivation, penalty methods, and adaptive strategies, demonstrated on various robotic control tasks.

Contribution

It introduces a unified derivation for multiple shooting DDP, improving robustness and convergence, and demonstrates its effectiveness in trajectory optimization and MPC for robots.

Findings

Improved convergence and robustness in trajectory optimization.

Effective application of MS-DDP in robotic control tasks.

Enhanced global convergence strategies for DDP.

Abstract

Differential Dynamic Programming (DDP) is an efficient computational tool for solving nonlinear optimal control problems. It was originally designed as a single shooting method and thus is sensitive to the initial guess supplied. This work considers the extension of DDP to multiple shooting (MS), improving its robustness to initial guesses. A novel derivation is proposed that accounts for the defect between shooting segments during the DDP backward pass, while still maintaining quadratic convergence locally. The derivation enables unifying multiple previous MS algorithms, and opens the door to many smaller algorithmic improvements. A penalty method is introduced to strategically control the step size, further improving the convergence performance. An adaptive merit function and a more reliable acceptance condition are employed for globalization. The effects of these improvements are benchmarked for trajectory optimization with a quadrotor, an acrobot, and a manipulator. MS-DDP is also demonstrated for use in Model Predictive Control (MPC) for dynamic jumping with a quadruped robot, showing its benefits over a single shooting approach.