Structure-aware reinforcement learning for node-overload protection in mobile edge computing

TL;DR

This paper introduces a structure-aware reinforcement learning approach called SALMUT for preventing node overloads in mobile edge computing, demonstrating efficiency and interpretability in simulations and testbeds.

Contribution

The work extends SALMUT to a discounted-cost setting for node overload protection in MEC, showing it is as effective as deep RL methods but more efficient and interpretable.

Findings

SALMUT achieves similar costs to PPO and A2C.

SALMUT requires significantly less training time.

The policy generated by SALMUT is easily interpretable.

Abstract

Mobile Edge Computing (MEC) refers to the concept of placing computational capability and applications at the edge of the network, providing benefits such as reduced latency in handling client requests, reduced network congestion, and improved performance of applications. The performance and reliability of MEC are degraded significantly when one or several edge servers in the cluster are overloaded. Especially when a server crashes due to the overload, it causes service failures in MEC. In this work, an adaptive admission control policy to prevent edge node from getting overloaded is presented. This approach is based on a recently-proposed low complexity RL (Reinforcement Learning) algorithm called SALMUT (Structure-Aware Learning for Multiple Thresholds), which exploits the structure of the optimal admission control policy in multi-class queues for an average-cost setting. We extend the framework to work for node overload-protection problem in a discounted-cost setting. The proposed solution is validated using several scenarios mimicking real-world deployments in two different settings - computer simulations and a docker testbed. Our empirical evaluations show that the total discounted cost incurred by SALMUT is similar to state-of-the-art deep RL algorithms such as PPO (Proximal Policy Optimization) and A2C (Advantage Actor Critic) but requires an order of magnitude less time to train, outputs easily interpretable policy, and can be deployed in an online manner.