Federated Control with Hierarchical Multi-Agent Deep Reinforcement Learning

TL;DR

This paper introduces a hierarchical multi-agent deep reinforcement learning framework with a semi-decentralized control structure, improving coordination among many agents for complex tasks.

Contribution

It proposes a novel hierarchical and semi-decentralized approach with a meta-controller to enhance multi-agent coordination and scalability.

Findings

Effective in simulated distributed scheduling tasks

Enables scalable coordination among many agents

Shows promising initial experimental results

Abstract

We present a framework combining hierarchical and multi-agent deep reinforcement learning approaches to solve coordination problems among a multitude of agents using a semi-decentralized model. The framework extends the multi-agent learning setup by introducing a meta-controller that guides the communication between agent pairs, enabling agents to focus on communicating with only one other agent at any step. This hierarchical decomposition of the task allows for efficient exploration to learn policies that identify globally optimal solutions even as the number of collaborating agents increases. We show promising initial experimental results on a simulated distributed scheduling problem.