Reinforcement Learning based Multi-connectivity Resource Allocation in Factory Automation Systems

TL;DR

This paper introduces a reinforcement learning-based resource allocation method for factory automation robots, enhancing reliability through multi-connectivity and distributed decision-making.

Contribution

It develops a distributed multi-agent reinforcement learning algorithm for joint user association and resource allocation in multi-connectivity URLLC systems.

Findings

Approaches centralized performance in reliability metrics.

Significantly outperforms single-connectivity schemes.

Demonstrates effectiveness in simulated factory scenarios.

Abstract

We propose joint user association, channel assignment and power allocation for mobile robot Ultra-Reliable and Low Latency Communications (URLLC) based on multi-connectivity and reinforcement learning. The mobile robots require control messages from the central guidance system at regular intervals. We use a two-phase communication scheme where robots can form multiple clusters. The robots in a cluster are close to each other and can have reliable Device-to-Device (D2D) communications. In Phase I, the APs transmit the combined payload of a cluster to the cluster leader within a latency constraint. The cluster leader broadcasts this message to its members in Phase II. We develop a distributed Multi-Agent Reinforcement Learning (MARL) algorithm for joint user association and resource allocation (RA) for Phase I. The cluster leaders use their local Channel State Information (CSI) to decide the APs for connection along with the sub-band and power level. The cluster leaders utilize multi-connectivity to connect to multiple APs to increase their reliability. The objective is to maximize the successful payload delivery probability for all robots. Illustrative simulation results indicate that the proposed scheme can approach the performance of the centralized algorithm and offer a substantial gain in reliability as compared to single-connectivity (when cluster leaders are able to connect to 1 AP).