Cooperative Multi-Agent Deep Reinforcement Learning Methods for UAV-aided Mobile Edge Computing Networks

TL;DR

This paper introduces a cooperative multi-agent deep reinforcement learning framework for UAV-assisted mobile edge computing networks, enabling decentralized decision-making for UAV trajectory, resource allocation, and task offloading, with scalable training and improved performance.

Contribution

It proposes a novel cooperative MADRL method with graph attention networks for scalable, decentralized MEC optimization in UAV-assisted networks, outperforming traditional approaches.

Findings

The approach achieves higher MEC performance than conventional methods.

It demonstrates scalability to various network sizes.

The method effectively coordinates UAV and IoT devices in decentralized settings.

Abstract

This paper presents a cooperative multi-agent deep reinforcement learning (MADRL) approach for unmmaned aerial vehicle (UAV)-aided mobile edge computing (MEC) networks. An UAV with computing capability can provide task offlaoding services to ground internet-of-things devices (IDs). With partial observation of the entire network state, the UAV and the IDs individually determine their MEC strategies, i.e., UAV trajectory, resource allocation, and task offloading policy. This requires joint optimization of decision-making process and coordination strategies among the UAV and the IDs. To address this difficulty, the proposed cooperative MADRL approach computes two types of action variables, namely message action and solution action, each of which is generated by dedicated actor neural networks (NNs). As a result, each agent can automatically encapsulate its coordination messages to enhance the MEC performance in the decentralized manner. The proposed actor structure is designed based on graph attention networks such that operations are possible regardless of the number of IDs. A scalable training algorithm is also proposed to train a group of NNs for arbitrary network configurations. Numerical results demonstrate the superiority of the proposed cooperative MADRL approach over conventional methods.