Distributed Differential Dynamic Programming Architectures for Large-Scale Multi-Agent Control

TL;DR

This paper introduces two scalable decentralized control architectures, ND-DDP and MD-DDP, for large-scale multi-agent systems, combining DDP and ADMM to improve efficiency and robustness in robotics applications.

Contribution

The paper presents two novel decentralized optimization frameworks, ND-DDP and MD-DDP, integrating DDP and ADMM for scalable multi-agent control.

Findings

Both frameworks are fully decentralized and parallelizable.

Simulation results demonstrate scalability to thousands of agents.

Hardware experiments confirm real-world applicability.

Abstract

In this paper, we propose two novel decentralized optimization frameworks for multi-agent nonlinear optimal control problems in robotics. The aim of this work is to suggest architectures that inherit the computational efficiency and scalability of Differential Dynamic Programming (DDP) and the distributed nature of the Alternating Direction Method of Multipliers (ADMM). In this direction, two frameworks are introduced. The first one called Nested Distributed DDP (ND-DDP), is a three-level architecture which employs ADMM for enforcing a consensus between all agents, an augmented Lagrangian layer for satisfying local constraints and DDP as each agent's optimizer. In the second approach, both consensus and local constraints are handled with ADMM, yielding a two-level architecture called Merged Distributed DDP (MD-DDP), which further reduces computational complexity. Both frameworks are fully decentralized since all computations are parallelizable among the agents and only local communication is necessary. Simulation results that scale up to thousands of vehicles and hundreds of drones verify the effectiveness of the methods. Superior scalability to large-scale systems against centralized DDP and centralized/decentralized sequential quadratic programming is also illustrated. Finally, hardware experiments on a multi-robot platform demonstrate the applicability of the proposed algorithms, while highlighting the importance of optimizing for feedback policies to increase robustness against uncertainty. A video including all results is available in https://youtu.be/tluvENcWldw.