Distributed Computation Offloading for Energy Provision Minimization in WP-MEC Networks with Multiple HAPs

TL;DR

This paper proposes a distributed deep reinforcement learning framework for optimizing energy-efficient computation offloading in multi-HAP WP-MEC networks, adapting to dynamic environments and minimizing long-term energy provision.

Contribution

It introduces a novel TMADO framework with a hierarchical multi-agent DRL approach for joint optimization in WP-MEC networks, addressing a complex non-convex problem.

Findings

TMADO outperforms baseline methods in energy minimization.

The framework effectively adapts to dynamic wireless channel conditions.

Distributed agents achieve near-optimal offloading and energy provisioning.

Abstract

This paper investigates a wireless powered mobile edge computing (WP-MEC) network with multiple hybrid access points (HAPs) in a dynamic environment, where wireless devices (WDs) harvest energy from radio frequency (RF) signals of HAPs, and then compute their computation data locally (i.e., local computing mode) or offload it to the chosen HAPs (i.e., edge computing mode). In order to pursue a green computing design, we formulate an optimization problem that minimizes the long-term energy provision of the WP-MEC network subject to the energy, computing delay and computation data demand constraints. The transmit power of HAPs, the duration of the wireless power transfer (WPT) phase, the offloading decisions of WDs, the time allocation for offloading and the CPU frequency for local computing are jointly optimized adapting to the time-varying generated computation data and wireless channels of WDs. To efficiently address the formulated non-convex mixed integer programming (MIP) problem in a distributed manner, we propose a Two-stage Multi-Agent deep reinforcement learning-based Distributed computation Offloading (TMADO) framework, which consists of a high-level agent and multiple low-level agents. The high-level agent residing in all HAPs optimizes the transmit power of HAPs and the duration of the WPT phase, while each low-level agent residing in each WD optimizes its offloading decision, time allocation for offloading and CPU frequency for local computing. Simulation results show the superiority of the proposed TMADO framework in terms of the energy provision minimization.