Joint Service Caching, Communication and Computing Resource Allocation in Collaborative MEC Systems: A DRL-based Two-timescale Approach

TL;DR

This paper introduces a DRL-based dual timescale approach for joint resource allocation in collaborative MEC systems, aiming to optimize QoS and reduce cache switching costs through innovative deep reinforcement learning techniques.

Contribution

It proposes a novel dual timescale DRL scheme, DGL-DDPG, combining genetic algorithms and LSTM-DDPG for efficient joint resource management in MEC systems.

Findings

Outperforms baseline algorithms in QoS metrics

Reduces cache switching costs significantly

Demonstrates effectiveness of dual timescale DRL approach

Abstract

Meeting the strict Quality of Service (QoS) requirements of terminals has imposed a signiffcant challenge on Multiaccess Edge Computing (MEC) systems, due to the limited multidimensional resources. To address this challenge, we propose a collaborative MEC framework that facilitates resource sharing between the edge servers, and with the aim to maximize the long-term QoS and reduce the cache switching cost through joint optimization of service caching, collaborative offfoading, and computation and communication resource allocation. The dual timescale feature and temporal recurrence relationship between service caching and other resource allocation make solving the problem even more challenging. To solve it, we propose a deep reinforcement learning (DRL)-based dual timescale scheme, called DGL-DDPG, which is composed of a short-term genetic algorithm (GA) and a long short-term memory network-based deep deterministic policy gradient (LSTM-DDPG). In doing so, we reformulate the optimization problem as a Markov decision process (MDP) where the small-timescale resource allocation decisions generated by an improved GA are taken as the states and input into a centralized LSTM-DDPG agent to generate the service caching decision for the large-timescale. Simulation results demonstrate that our proposed algorithm outperforms the baseline algorithms in terms of the average QoS and cache switching cost.