Robust synchronization and policy adaptation for networked heterogeneous agents

TL;DR

This paper introduces a robust adaptive control method combining reinforcement learning and input saturation handling to achieve synchronization in heterogeneous nonlinear agent networks with uncertainties.

Contribution

It presents the DMSAC-RL algorithm that adaptively adjusts policies for heterogeneous agents, ensuring synchronization despite uncertainties and input saturation.

Findings

Synchronization error is proven to be uniformly ultimately bounded.

Numerical simulations validate the theoretical stability and performance.

The method improves empirical policy performance in complex networked systems.

Abstract

We propose a robust adaptive online synchronization method for leader-follower networks of nonlinear heterogeneous agents with system uncertainties and input magnitude saturation. Synchronization is achieved using a Distributed input Magnitude Saturation Adaptive Control with Reinforcement Learning (DMSAC-RL), which improves the empirical performance of policies trained on off-the-shelf models using Reinforcement Learning (RL) strategies. The leader observes the performance of a reference model, and followers observe the states and actions of the agents they are connected to, but not the reference model. The leader and followers may differ from the reference model in which the RL control policy was trained. DMSAC-RL uses an internal loop that adjusts the learned policy for the agents in the form of augmented input to solve the distributed control problem, including input-matched uncertainty parameters. We show that the synchronization error of the heterogeneous network is Uniformly Ultimately Bounded (UUB). Numerical analysis of a network of Multiple Input Multiple Output (MIMO) systems supports our theoretical findings.