QMA: A Resource-efficient, Q-Learning-based Multiple Access Scheme for the IIoT

TL;DR

QMA is a resource-efficient, Q-learning-based multiple access scheme for IIoT that dynamically adapts transmission times to improve reliability and throughput without extra overhead, especially in dense networks.

Contribution

The paper introduces QMA, a novel contention-based access scheme using Q-learning to learn traffic patterns and improve performance in IIoT networks.

Findings

QMA effectively solves the hidden node problem without RTS/CTS overhead.

QMA significantly increases reliability and throughput in high-load networks.

QMA demonstrates scalability and improved performance in simulations and testbed experiments.

Abstract

Contention-based wireless channel access methods like CSMA and ALOHA paved the way for the rise of the Internet of Things in industrial applications (IIoT). However, to cope with increasing demands for reliability and throughput, several mostly TDMA-based protocols like IEEE 802.15.4 and its extensions were proposed. Nonetheless, many of these IIoT-protocols still require contention-based communication, e.g., for slot allocation and broadcast transmission. In many cases, subtle but hidden patterns characterize this secondary traffic. Present contention-based protocols are unaware of these hidden patterns and can therefore not exploit this information. Especially in dense networks, they often do not provide sufficient reliability for primary traffic, e.g., they are unable to allocate transmission slots in time. In this paper, we propose QMA, a contention-based multiple access scheme based on Q-learning, which dynamically adapts transmission times to avoid collisions by learning patterns in the contention-based traffic. QMA is designed to be resource-efficient and targets small embedded devices. We show that QMA solves the hidden node problem without the additional overhead of RTS / CTS messages and verify the behaviour of QMA in the FIT IoT-LAB testbed. Finally, QMA's scalability is studied by simulation, where it is used for GTS allocation in IEEE 802.15.4 DSME. Results show that QMA considerably increases reliability and throughput in comparison to CSMA/CA, especially in networks with a high load.