Optimizing Trajectories with Closed-Loop Dynamic SQP

TL;DR

This paper introduces a novel dynamic programming approach to optimize trajectories with constraints using a hybrid SQP method that incorporates closed-loop policies, significantly improving convergence speed over traditional open-loop methods.

Contribution

It develops a new dynamic programming-based recursion for constrained problems and a hybrid SQP algorithm with parallelized feedback gain computation.

Findings

Significant convergence speed improvements over standard open-loop SQP.

Effective computation of feedback gains for constrained trajectory optimization.

Validated on challenging benchmarks with promising results.

Abstract

Indirect trajectory optimization methods such as Differential Dynamic Programming (DDP) have found considerable success when only planning under dynamic feasibility constraints. Meanwhile, nonlinear programming (NLP) has been the state-of-the-art approach when faced with additional constraints (e.g., control bounds, obstacle avoidance). However, a na$\"i$ve implementation of NLP algorithms, e.g., shooting-based sequential quadratic programming (SQP), may suffer from slow convergence -- caused from natural instabilities of the underlying system manifesting as poor numerical stability within the optimization. Re-interpreting the DDP closed-loop rollout policy as a sensitivity-based correction to a second-order search direction, we demonstrate how to compute analogous closed-loop policies (i.e., feedback gains) for constrained problems. Our key theoretical result introduces a novel dynamic programming-based constraint-set recursion that augments the canonical "cost-to-go" backward pass. On the algorithmic front, we develop a hybrid-SQP algorithm incorporating DDP-style closed-loop rollouts, enabled via efficient parallelized computation of the feedback gains. Finally, we validate our theoretical and algorithmic contributions on a set of increasingly challenging benchmarks, demonstrating significant improvements in convergence speed over standard open-loop SQP.