Hierarchical Reinforcement Learning for Optimal Agent Grouping in Cooperative Systems

TL;DR

This paper introduces a hierarchical reinforcement learning framework for optimal agent grouping in cooperative multi-agent systems, combining CTDE, permutation-invariant networks, and option-critic algorithms for scalable and effective coordination.

Contribution

It proposes a novel hierarchical RL approach that jointly learns agent grouping and policies, enhancing coordination in cooperative multi-agent systems.

Findings

Effective agent grouping achieved in cooperative tasks

Scalable learning with CTDE paradigm

Improved coordination through permutation-invariant networks

Abstract

This paper presents a hierarchical reinforcement learning (RL) approach to address the agent grouping or pairing problem in cooperative multi-agent systems. The goal is to simultaneously learn the optimal grouping and agent policy. By employing a hierarchical RL framework, we distinguish between high-level decisions of grouping and low-level agents' actions. Our approach utilizes the CTDE (Centralized Training with Decentralized Execution) paradigm, ensuring efficient learning and scalable execution. We incorporate permutation-invariant neural networks to handle the homogeneity and cooperation among agents, enabling effective coordination. The option-critic algorithm is adapted to manage the hierarchical decision-making process, allowing for dynamic and optimal policy adjustments.