A Distributed Actor-Critic Algorithm for Fixed-Time Consensus in Nonlinear Multi-Agent Systems

TL;DR

This paper introduces a reinforcement learning-based distributed control method that guarantees fixed-time consensus in nonlinear multi-agent systems with unknown dynamics, using output-only communication and a novel adaptation mechanism.

Contribution

It develops a fixed-time adaptive actor-critic reinforcement learning algorithm for nonlinear multi-agent systems with directed communication graphs, ensuring convergence within a pre-specified time.

Findings

Achieves fixed-time consensus regardless of initial conditions.

Demonstrates robustness to unknown nonlinearities and disturbances.

Outperforms existing methods in convergence speed and robustness.

Abstract

This paper proposes a reinforcement learning (RL)-based backstepping control strategy to achieve fixed time consensus in nonlinear multi-agent systems with strict feedback dynamics. Agents exchange only output information with their neighbors over a directed communication graph, without requiring full state measurements or symmetric communication. Achieving fixed time consensus, where convergence occurs within a pre-specified time bound that is independent of initial conditions is faced with significant challenges due to the presence of unknown nonlinearities, inter-agent couplings, and external disturbances. This work addresses these challenges by integrating actor critic reinforcement learning with a novel fixed time adaptation mechanism. Each agent employs an actor critic architecture supported by two estimator networks designed to handle system uncertainties and unknown perturbations. The adaptation laws are developed to ensure that all agents track the leader within a fixed time regardless of their initial conditions. The consensus and tracking errors are guaranteed to converge to a small neighborhood of the origin, with the convergence radius adjustable through control parameters. Simulation results demonstrate the effectiveness of the proposed approach and highlight its advantages over state-of-the-art methods in terms of convergence speed and robustness.