Reinforcement Learning-Enabled Dynamic Code Assignment for Ultra-Dense IoT Networks: A NOMA-Based Approach to Massive Device Connectivity

TL;DR

This paper proposes a reinforcement learning approach for dynamic code assignment in ultra-dense IoT networks to improve throughput, energy efficiency, and fairness, addressing interference issues in NOMA-based systems.

Contribution

It introduces a novel RL model with joint optimization for IoT-NOMA networks, including two algorithms for adaptive code assignment, enhancing network performance in smart city scenarios.

Findings

NPG achieves 11.6% higher throughput in smart city conditions.

Energy efficiency improves by 15.8% with dynamic code assignment.

Performance drops in industrial settings, indicating need for additional measures.

Abstract

Ultra-dense IoT networks require an effective non-orthogonal multiple access (NOMA) scheme, yet they experience intense interference because of fixed code assignment. We suggest a reinforcement learning (RL) model of dynamic Gold code assignment in IoT-NOMA networks. Our Markov Decision Process which is IoT aware is a joint optimization of throughput, energy efficiency, and fairness. Two RL algorithms are created, including Natural Policy Gradient (NPG) to learn stable discrete actions and Deep Deterministic Policy Gradient (DDPG) with continuous code embedding. Under smart city conditions, NPG can attain throughput of 11.6% and energy efficiency of 15.8 likewise superior to its performance with a static allocation. Nonetheless, the performance is worse in organized industrial settings, and the reliability is minimal (0-2%), which points to the fact that dynamic code assignment is not a sufficient measure of ultra-reliable IoT and needs to be supplemented by power control or retransmission schemes. The work offers a basis to the RL-based resource allocation in massive IoT network.