Scalable UAV Multi-Hop Networking via Multi-Agent Reinforcement Learning with Large Language Models

TL;DR

This paper introduces MRLMN, a scalable framework combining multi-agent reinforcement learning and large language models to optimize UAV multi-hop networks in disaster scenarios, improving coverage and robustness.

Contribution

The paper presents a novel integration of LLMs with MARL for UAV networking, including a grouping strategy, reward decomposition, behavioral constraints, and knowledge distillation techniques.

Findings

Significant performance improvements over MAPPO baseline.

Enhanced network coverage and communication quality.

Effective scalability in large dynamic environments.

Abstract

In disaster scenarios, establishing robust emergency communication networks is critical, and unmanned aerial vehicles (UAVs) offer a promising solution to rapidly restore connectivity. However, organizing UAVs to form multi-hop networks in large-scale dynamic environments presents significant challenges, including limitations in algorithmic scalability and the vast exploration space required for coordinated decision-making. To address these issues, we propose MRLMN, a novel framework that integrates multi-agent reinforcement learning (MARL) and large language models (LLMs) to jointly optimize UAV agents toward achieving optimal networking performance. The framework incorporates a grouping strategy with reward decomposition to enhance algorithmic scalability and balance decision-making across UAVs. In addition, behavioral constraints are applied to selected key UAVs to improve the robustness of the network. Furthermore, the framework integrates LLM agents, leveraging knowledge distillation to transfer their high-level decision-making capabilities to MARL agents. This enhances both the efficiency of exploration and the overall training process. In the distillation module, a Hungarian algorithm-based matching scheme is applied to align the decision outputs of the LLM and MARL agents and define the distillation loss. Extensive simulation results validate the effectiveness of our approach, demonstrating significant improvements in network performance over the MAPPO baseline and other comparison methods, including enhanced coverage and communication quality.