Data-Based Optimal Control of Multi-Agent Systems: A Reinforcement Learning Design Approach

TL;DR

This paper presents a data-driven reinforcement learning approach for optimal consensus control in heterogeneous multi-agent systems, eliminating the need for system models by using I/O data and value iteration.

Contribution

It introduces a novel data-based error estimator and an I/O Q-learning algorithm to solve the optimal consensus problem without relying on system models.

Findings

The proposed method effectively achieves optimal consensus control.

The approach does not require explicit system models.

Numerical results demonstrate the algorithm's effectiveness.

Abstract

This paper studies optimal consensus tracking problem of heterogeneous linear multi-agent systems. By introducing tracking error dynamics, the optimal tracking problem is reformulated as finding a Nash-equilibrium solution of a multi-player games, which can be done by solving associated coupled Hamilton-Jacobi (HJ) equations. A data-based error estimator is designed to obtain the data-based control for the multi-agent systems. Using the quadratic functional to approximate the every agent's value function, we can obtain the optimal cooperative control by input-output (I/O) $Q$-learning algorithm with value iteration technique in the least-square sense. The control law solves the optimal consensus problem for multi-agent systems with measured input-output information, and does not rely on the model of multi-agent systems. A numerical example is provided to illustrate the effectiveness of the proposed algorithm.