# Differential Dynamic Programming for time-delayed systems

## TL;DR

This paper extends Differential Dynamic Programming (DDP) to handle systems with multiple time-delays, enabling optimal control for more complex, delay-influenced dynamical systems, demonstrated on chemical reactors and neural network models.

## Contribution

The paper introduces a novel extension of DDP to systems with multiple time-delays, broadening its applicability to richer models including neural networks.

## Key findings

- Successfully applied to a two-tank reactor system.
- Effective control of a recurrent neural network model of an inverted pendulum.
- Demonstrates real-time feasible trajectory optimization for delayed systems.

## Abstract

Trajectory optimization considers the problem of deciding how to control a dynamical system to move along a trajectory which minimizes some cost function. Differential Dynamic Programming (DDP) is an optimal control method which utilizes a second-order approximation of the problem to find the control. It is fast enough to allow real-time control and has been shown to work well for trajectory optimization in robotic systems. Here we extend classic DDP to systems with multiple time-delays in the state. Being able to find optimal trajectories for time-delayed systems with DDP opens up the possibility to use richer models for system identification and control, including recurrent neural networks with multiple timesteps in the state. We demonstrate the algorithm on a two-tank continuous stirred tank reactor. We also demonstrate the algorithm on a recurrent neural network trained to model an inverted pendulum with position information only.