Optimal Flow Admission Control in Edge Computing via Safe Reinforcement Learning

TL;DR

This paper introduces DR-CPO, a safe reinforcement learning algorithm for optimal flow admission control in edge computing, improving reward and convergence speed while managing heterogeneous resources effectively.

Contribution

The paper develops a novel primal-dual safe reinforcement learning method for optimal flow admission, addressing state-space complexity and decentralization in edge computing environments.

Findings

DR-CPO achieves 15% higher reward than existing DRL methods.

DR-CPO requires only 50% of the learning episodes for convergence.

Effective load balancing further enhances system performance.

Abstract

With the uptake of intelligent data-driven applications, edge computing infrastructures necessitate a new generation of admission control algorithms to maximize system performance under limited and highly heterogeneous resources. In this paper, we study how to optimally select information flows which belong to different classes and dispatch them to multiple edge servers where deployed applications perform flow analytic tasks. The optimal policy is obtained via constrained Markov decision process (CMDP) theory accounting for the demand of each edge application for specific classes of flows, the constraints on computing capacity of edge servers and of the access network. We develop DR-CPO, a specialized primal-dual Safe Reinforcement Learning (SRL) method which solves the resulting optimal admission control problem by reward decomposition. DR-CPO operates optimal decentralized control and mitigates effectively state-space explosion while preserving optimality. Compared to existing Deep Reinforcement Learning (DRL) solutions, extensive results show that DR-CPO achieves 15\% higher reward on a wide variety of environments, while requiring on average only 50\% of the amount of learning episodes to converge. Finally, we show how to match DR-CPO and load-balancing to dispatch optimally information streams to available edge servers and further improve system performance.