Meta-CPR: Generalize to Unseen Large Number of Agents with Communication Pattern Recognition Module

TL;DR

Meta-CPR introduces a meta reinforcement learning framework with a Communication Pattern Recognition module that enables multi-agent systems to adapt to unseen larger groups and dynamic changes in agent numbers, improving real-world applicability.

Contribution

The paper proposes a novel meta-RL framework with a CPR module that generalizes to unseen agent counts and handles dynamic agent environments.

Findings

Framework generalizes to larger unseen agent groups

Allows agent number to change between episodes

Ablation study confirms CPR effectiveness

Abstract

Designing an effective communication mechanism among agents in reinforcement learning has been a challenging task, especially for real-world applications. The number of agents can grow or an environment sometimes needs to interact with a changing number of agents in real-world scenarios. To this end, a multi-agent framework needs to handle various scenarios of agents, in terms of both scales and dynamics, for being practical to real-world applications. We formulate the multi-agent environment with a different number of agents as a multi-tasking problem and propose a meta reinforcement learning (meta-RL) framework to tackle this problem. The proposed framework employs a meta-learned Communication Pattern Recognition (CPR) module to identify communication behavior and extract information that facilitates the training process. Experimental results are poised to demonstrate that the proposed framework (a) generalizes to an unseen larger number of agents and (b) allows the number of agents to change between episodes. The ablation study is also provided to reason the proposed CPR design and show such design is effective.