Combining Contention-Based Spectrum Access and Adaptive Modulation using Deep Reinforcement Learning

TL;DR

This paper introduces a deep reinforcement learning method for decentralized spectrum access and adaptive modulation in unlicensed spectrum, significantly improving throughput and fairness in cellular networks.

Contribution

It presents a novel distributed deep RL algorithm for contention and adaptive modulation, outperforming traditional methods and demonstrating scalability to large networks.

Findings

Higher proportional fairness reward than genie-aided methods

Improved sum and peak throughput in simulations

Scalable performance in large indoor and outdoor networks

Abstract

The use of unlicensed spectrum for cellular systems to mitigate spectrum scarcity has led to the development of intelligent adaptive approaches to spectrum access that improve upon traditional carrier sensing and listen-before-talk methods. We study decentralized contention-based medium access for base stations (BSs) of a single Radio Access Technology (RAT) operating on unlicensed shared spectrum. We devise a distributed deep reinforcement learning-based algorithm for both contention and adaptive modulation, modelled on a two state Markov decision process, that attempts to maximize a network-wide downlink throughput objective. Empirically, we find the (proportional fairness) reward accumulated by a policy gradient approach to be significantly higher than even a genie-aided adaptive energy detection threshold. Our approaches are further validated by improved sum and peak throughput. The scalability of our approach to large networks is demonstrated via an improved cumulative reward earned on both indoor and outdoor layouts with a large number of BSs.