Throughput and Latency in the Distributed Q-Learning Random Access mMTC Networks

TL;DR

This paper introduces a distributed Q-learning approach for random access in massive machine-type communication networks, improving throughput and latency trade-offs by prioritizing devices with more packets to transmit.

Contribution

It proposes a novel distributed packet-based Q-learning method that adjusts rewards based on remaining packets, outperforming existing independent and collaborative techniques.

Findings

Achieves better throughput-latency trade-off in simulations.

Reduces payload bits compared to collaborative Q-learning.

Demonstrates effectiveness in practical mMTC scenarios.

Abstract

In mMTC mode, with thousands of devices trying to access network resources sporadically, the problem of random access (RA) and collisions between devices that select the same resources becomes crucial. A promising approach to solve such an RA problem is to use learning mechanisms, especially the Q-learning algorithm, where the devices learn about the best time-slot periods to transmit through rewards sent by the central node. In this work, we propose a distributed packet-based learning method by varying the reward from the central node that favors devices having a larger number of remaining packets to transmit. Our numerical results indicated that the proposed distributed packet-based Q-learning method attains a much better throughput-latency trade-off than the alternative independent and collaborative techniques in practical scenarios of interest. In contrast, the number of payload bits of the packet-based technique is reduced regarding the collaborative Q-learning RA technique for achieving the same normalized throughput.