Pontryagin-Bellman Differential Dynamic Programming for Low-Thrust Trajectory Optimization with Path Constraints

TL;DR

This paper presents PDDP, a novel algorithm combining Pontryagin Minimum Principle with Differential Dynamic Programming to efficiently solve constrained nonlinear optimal control problems, especially for low-thrust spacecraft trajectories.

Contribution

The paper introduces PDDP, integrating PMP into DDP with advanced constraint handling, improving robustness and efficiency for complex trajectory optimization tasks.

Findings

PDDP effectively handles state and terminal constraints in low-thrust trajectory optimization.

The method reduces sensitivity to initial guesses compared to classical indirect methods.

PDDP enables long-duration, multi-spiral trajectory design with fewer variables.

Abstract

We introduce a new algorithm to solve constrained nonlinear optimal control problem, with an emphasis on low-thrust trajectory in highly nonlinear dynamics. The algorithm, dubbed Pontryagin-Bellman Differential Dynamic Programming (PDDP), is the result of the incorporation of Pontryagin Minimum Principle (PMP) into the Differential Dynamic Programming (DDP) formulation. Unlike traditional indirect methods that rely on first-order shooting techniques to determine the optimal costate, PDDP optimizes the costates using a null-space trust-region method, solving a series of quadratic subproblems derived from first- and second-order sensitivities. Terminal equality constraints are handled via a general augmented Lagrangian method, while state-path constraints are enforced using a quadratic penalty approach. The resulting solution method represents a significant improvement over classical indirect methods, which are known to be ineffective for state-constrained problems. The use of DDP to optimize the costate partially reduces the inherent sensitivities of indirect methods, thereby improving the robustness to poor initial guess. Finally, the incorporation of PMP within the DDP formulation enables the design of long-duration, multi-spiral low-thrust trajectories with fewer optimization variables compared to classical DDP formulations. Although the paper emphasizes the spacecraft low-thrust trajectory optimization problem, the theoretical foundations are general and applicable to any control-affine optimal control problem.