CLARA: A Constrained Reinforcement Learning Based Resource Allocation Framework for Network Slicing

TL;DR

This paper introduces CLARA, a novel constrained reinforcement learning framework for dynamic resource allocation in network slicing, addressing the complexity of traditional methods by formulating the problem as a CMDP without prior models.

Contribution

The paper proposes CLARA, a model-free constrained reinforcement learning algorithm utilizing adaptive interior-point policy optimization for efficient resource management in network slicing.

Findings

CLARA outperforms baseline methods in resource allocation efficiency.

It guarantees service demand satisfaction through constrained optimization.

The approach effectively handles complex, model-free network slicing scenarios.

Abstract

As mobile networks proliferate, we are experiencing a strong diversification of services, which requires greater flexibility from the existing network. Network slicing is proposed as a promising solution for resource utilization in 5G and future networks to address this dire need. In network slicing, dynamic resource orchestration and network slice management are crucial for maximizing resource utilization. Unfortunately, this process is too complex for traditional approaches to be effective due to a lack of accurate models and dynamic hidden structures. We formulate the problem as a Constrained Markov Decision Process (CMDP) without knowing models and hidden structures. Additionally, we propose to solve the problem using CLARA, a Constrained reinforcement LeArning based Resource Allocation algorithm. In particular, we analyze cumulative and instantaneous constraints using adaptive interior-point policy optimization and projection layer, respectively. Evaluations show that CLARA clearly outperforms baselines in resource allocation with service demand guarantees.