Self-Play Ensemble Q-learning enabled Resource Allocation for Network Slicing

TL;DR

This paper introduces self-play ensemble Q-learning for network slicing in 5G, improving resource allocation by reducing overestimation and enhancing robustness against adversarial attacks, outperforming traditional RL methods.

Contribution

The paper proposes a novel self-play ensemble Q-learning approach that enhances RL-based resource allocation in 5G network slicing, addressing overestimation and robustness issues.

Findings

Self-play ensemble Q-learning reduces latency by 21.92%.

It improves throughput by 24.22%.

It is more robust against malicious users compared to baseline methods.

Abstract

In 5G networks, network slicing has emerged as a pivotal paradigm to address diverse user demands and service requirements. To meet the requirements, reinforcement learning (RL) algorithms have been utilized widely, but this method has the problem of overestimation and exploration-exploitation trade-offs. To tackle these problems, this paper explores the application of self-play ensemble Q-learning, an extended version of the RL-based technique. Self-play ensemble Q-learning utilizes multiple Q-tables with various exploration-exploitation rates leading to different observations for choosing the most suitable action for each state. Moreover, through self-play, each model endeavors to enhance its performance compared to its previous iterations, boosting system efficiency, and decreasing the effect of overestimation. For performance evaluation, we consider three RL-based algorithms; self-play ensemble Q-learning, double Q-learning, and Q-learning, and compare their performance under different network traffic. Through simulations, we demonstrate the effectiveness of self-play ensemble Q-learning in meeting the diverse demands within 21.92% in latency, 24.22% in throughput, and 23.63\% in packet drop rate in comparison with the baseline methods. Furthermore, we evaluate the robustness of self-play ensemble Q-learning and double Q-learning in situations where one of the Q-tables is affected by a malicious user. Our results depicted that the self-play ensemble Q-learning method is more robust against adversarial users and prevents a noticeable drop in system performance, mitigating the impact of users manipulating policies.