A Distributed Neural Linear Thompson Sampling Framework to Achieve URLLC in Industrial IoT

TL;DR

This paper introduces DISNETS, a distributed reinforcement learning framework that enables IIoT devices to autonomously optimize uplink resource scheduling, significantly improving URLLC performance without extra communication overhead.

Contribution

The paper presents a novel multi-agent neural linear Thompson sampling framework that combines centralized feedback with autonomous device decision-making for URLLC in IIoT.

Findings

DISNETS outperforms baseline scheduling methods in URLLC scenarios.

The framework reduces collision rates and latency in uplink communications.

Distributed learning enables scalable and efficient resource allocation in IIoT networks.

Abstract

Industrial Internet of Things (IIoT) networks will provide Ultra-Reliable Low-Latency Communication (URLLC) to support critical processes underlying the production chains. However, standard protocols for allocating wireless resources may not optimize the latency-reliability trade-off, especially for uplink communication. For example, centralized grant-based scheduling can ensure almost zero collisions, but introduces delays in the way resources are requested by the User Equipments (UEs) and granted by the gNB. In turn, distributed scheduling (e.g., based on random access), in which UEs autonomously choose the resources for transmission, may lead to potentially many collisions especially when the traffic increases. In this work we propose DIStributed combinatorial NEural linear Thompson Sampling (DISNETS), a novel scheduling framework that combines the best of the two worlds. By leveraging a feedback signal from the gNB and reinforcement learning, the UEs are trained to autonomously optimize their uplink transmissions by selecting the available resources to minimize the number of collisions, without additional message exchange to/from the gNB. DISNETS is a distributed, multi-agent adaptation of the Neural Linear Thompson Sampling (NLTS) algorithm, which has been further extended to admit multiple parallel actions. We demonstrate the superior performance of DISNETS in addressing URLLC in IIoT scenarios compared to other baselines.