Collision Resolution with Deep Reinforcement Learning for Random Access in Machine-Type Communication

TL;DR

This paper introduces a multi-agent deep reinforcement learning approach to collision resolution in machine-type communication, outperforming traditional exponential backoff schemes in terms of throughput, delay, and collision rate.

Contribution

It proposes a novel DQN-based collision resolution scheme with parameter sharing for all devices, tailored for MTC traffic, reducing signaling overhead and improving performance.

Findings

DQN-RA outperforms exponential backoff in simulations

The scheme achieves better throughput, delay, and collision rate balance

Effective for up to 500 machine-type devices

Abstract

Grant-free random access (RA) techniques are suitable for machine-type communication (MTC) networks but they need to be adaptive to the MTC traffic, which is different from the human-type communication. Conventional RA protocols such as exponential backoff (EB) schemes for slotted-ALOHA suffer from a high number of collisions and they are not directly applicable to the MTC traffic models. In this work, we propose to use multi-agent deep Q-network (DQN) with parameter sharing to find a single policy applied to all machine-type devices (MTDs) in the network to resolve collisions. Moreover, we consider binary broadcast feedback common to all devices to reduce signalling overhead. We compare the performance of our proposed DQN-RA scheme with EB schemes for up to 500 MTDs and show that the proposed scheme outperforms EB policies and provides a better balance between throughput, delay and collision rate