Group-Agent Reinforcement Learning

TL;DR

This paper introduces a new group-agent reinforcement learning framework where geographically distributed agents share knowledge without direct cooperation, enhancing scalability and stability in RL tasks.

Contribution

It formulates the first group-agent RL system, distinct from multi-agent RL, and proposes the DDAL framework for effective distributed learning.

Findings

DDAL achieves stable training performance.

DDAL demonstrates good scalability.

Group-agent RL outperforms baseline methods.

Abstract

It can largely benefit the reinforcement learning (RL) process of each agent if multiple geographically distributed agents perform their separate RL tasks cooperatively. Different from multi-agent reinforcement learning (MARL) where multiple agents are in a common environment and should learn to cooperate or compete with each other, in this case each agent has its separate environment and only communicates with others to share knowledge without any cooperative or competitive behaviour as a learning outcome. In fact, this scenario exists widely in real life whose concept can be utilised in many applications, but is not well understood yet and not well formulated. As the first effort, we propose group-agent system for RL as a formulation of this scenario and the third type of RL system with respect to single-agent and multi-agent systems. We then propose a distributed RL framework called DDAL (Decentralised Distributed Asynchronous Learning) designed for group-agent reinforcement learning (GARL). We show through experiments that DDAL achieved desirable performance with very stable training and has good scalability.