Approximate optimal cooperative decentralized control for consensus in a topological network of agents with uncertain nonlinear dynamics

TL;DR

This paper develops a decentralized adaptive control method combining graph theory and reinforcement learning to achieve near-optimal consensus in multi-agent systems with uncertain nonlinear dynamics, using local communication and online estimation.

Contribution

It introduces a novel actor-critic-identifier architecture with a nonlinear state derivative estimator for real-time control of uncertain multi-agent systems.

Findings

Effective decentralized control with local communication

Online estimation improves control accuracy

Achieves approximate optimal consensus

Abstract

Efforts in this paper seek to combine graph theory with adaptive dynamic programming (ADP) as a reinforcement learning (RL) framework to determine forward-in-time, real-time, approximate optimal controllers for distributed multi-agent systems with uncertain nonlinear dynamics. A decentralized continuous time-varying control strategy is proposed, using only local communication feedback from two-hop neighbors on a communication topology that has a spanning tree. An actor-critic-identifier architecture is proposed that employs a nonlinear state derivative estimator to estimate the unknown dynamics online and uses the estimate thus obtained for value function approximation.