Transmit Power Pool Design for Grant-Free NOMA-IoT Networks via Deep Reinforcement Learning

TL;DR

This paper introduces a deep reinforcement learning-based method to design a transmit power pool for grant-free NOMA IoT networks, enabling efficient open-loop power control and improving throughput over traditional methods.

Contribution

It proposes a novel multi-agent deep Q-network approach with a prototype power pool for power allocation in GF-NOMA, addressing the lack of closed-loop control.

Findings

Double DQN algorithm effectively finds optimal power levels.

The proposed method converges faster than online learning approaches.

System outperforms fixed power and orthogonal access in throughput.

Abstract

Grant-free non-orthogonal multiple access (GF-NOMA) is a potential multiple access framework for short-packet internet-of-things (IoT) networks to enhance connectivity. However, the resource allocation problem in GF-NOMA is challenging due to the absence of closed-loop power control. We design a prototype of transmit power pool (PP) to provide open-loop power control. IoT users acquire their transmit power in advance from this prototype PP solely according to their communication distances. Firstly, a multi-agent deep Q-network (DQN) aided GF-NOMA algorithm is proposed to determine the optimal transmit power levels for the prototype PP. More specifically, each IoT user acts as an agent and learns a policy by interacting with the wireless environment that guides them to select optimal actions. Secondly, to prevent the Q-learning model overestimation problem, double DQN based GF-NOMA algorithm is proposed. Numerical results confirm that the double DQN based algorithm finds out the optimal transmit power levels that form the PP. Comparing with the conventional online learning approach, the proposed algorithm with the prototype PP converges faster under changing environments due to limiting the action space based on previous learning. The considered GF-NOMA system outperforms the networks with fixed transmission power, namely all the users have the same transmit power and the traditional GF with orthogonal multiple access techniques, in terms of throughput.