Federated Multi-Agent Deep Reinforcement Learning for Dynamic and Flexible 3D Operation of 5G Multi-MAP Networks

TL;DR

This paper introduces a federated multi-agent deep reinforcement learning framework for dynamic 3D placement of UAV-based MAPs in 5G networks, optimizing network reconfiguration and backhaul connectivity.

Contribution

It presents a novel hierarchical architecture with federated DRL for cooperative MAP placement, considering IAB constraints and joint optimization of placement and backhaul connectivity.

Findings

Effective dynamic MAP placement with cooperative DRL.

Federated training improves generalization and reduces complexity.

Optimized placement enhances network reconfiguration and backhaul connectivity.

Abstract

This paper addresses the efficient management of Mobile Access Points (MAPs), which are Unmanned Aerial Vehicles (UAV), in 5G networks. We propose a two-level hierarchical architecture, which dynamically reconfigures the network while considering Integrated Access-Backhaul (IAB) constraints. The high-layer decision process determines the number of MAPs through consensus, and we develop a joint optimization process to account for co-dependence in network self-management. In the low-layer, MAPs manage their placement using a double-attention based Deep Reinforcement Learning (DRL) model that encourages cooperation without retraining. To improve generalization and reduce complexity, we propose a federated mechanism for training and sharing one placement model for every MAP in the low-layer. Additionally, we jointly optimize the placement and backhaul connectivity of MAPs using a multi-objective reward function, considering the impact of varying MAP placement on wireless backhaul connectivity.