Two-Hop Age of Information Scheduling for Multi-UAV Assisted Mobile Edge Computing: FRL vs MADDPG

TL;DR

This paper explores optimizing the freshness of information in UAV-assisted mobile edge computing networks using deep reinforcement learning, comparing federated RL with MADDPG, and demonstrates significant performance improvements.

Contribution

It introduces a federated reinforcement learning approach for two-hop AoI minimization in UAV-assisted MEC, outperforming decentralized MADDPG methods.

Findings

FRL outperforms MADDPG by about 38% in AoI minimization.

The proposed method effectively manages UAV trajectories and task freshness.

Joint processing at UAVs and BS reduces network AoI.

Abstract

In this work, we adopt the emerging technology of mobile edge computing (MEC) in the Unmanned aerial vehicles (UAVs) for communication-computing systems, to optimize the age of information (AoI) in the network. We assume that tasks are processed jointly on UAVs and BS to enhance edge performance with limited connectivity and computing. Using UAVs and BS jointly with MEC can reduce AoI on the network. To maintain the freshness of the tasks, we formulate the AoI minimization in two-hop communication framework, the first hop at the UAVs and the second hop at the BS. To approach the challenge, we optimize the problem using a deep reinforcement learning (DRL) framework, called federated reinforcement learning (FRL). In our network we have two types of agents with different states and actions but with the same policy. Our FRL enables us to handle the two-step AoI minimization and UAV trajectory problems. In addition, we compare our proposed algorithm, which has a centralized processing unit to update the weights, with fully decentralized multi-agent deep deterministic policy gradient (MADDPG), which enhances the agent's performance. As a result, the suggested algorithm outperforms the MADDPG by about 38\%