Reinforcement Learning-Based Policy Optimisation For Heterogeneous Radio Access

TL;DR

This paper introduces a reinforcement learning approach to optimize resource sharing in heterogeneous wireless networks, improving IoT latency and maintaining broadband throughput and energy efficiency.

Contribution

It proposes a double Q-Learning based policy for IoT transmission optimization in grant-free access, applicable to both RAN slicing and sharing scenarios.

Findings

RL policies improve IoT latency performance

RAN sharing enhances energy efficiency at low traffic

RAN slicing is preferable at high IoT traffic

Abstract

Flexible and efficient wireless resource sharing across heterogeneous services is a key objective for future wireless networks. In this context, we investigate the performance of a system where latency-constrained internet-of-things (IoT) devices coexist with a broadband user. The base station adopts a grant-free access framework to manage resource allocation, either through orthogonal radio access network (RAN) slicing or by allowing shared access between services. For the IoT users, we propose a reinforcement learning (RL) approach based on double Q-Learning (QL) to optimise their repetition-based transmission strategy, allowing them to adapt to varying levels of interference and meet a predefined latency target. We evaluate the system's performance in terms of the cumulative distribution function of IoT users' latency, as well as the broadband user's throughput and energy efficiency (EE). Our results show that the proposed RL-based access policies significantly enhance the latency performance of IoT users in both RAN Slicing and RAN Sharing scenarios, while preserving desirable broadband throughput and EE. Furthermore, the proposed policies enable RAN Sharing to be energy-efficient at low IoT traffic levels, and RAN Slicing to be favourable under high IoT traffic.