Deep Reinforcement Learning for Radio Resource Allocation in NOMA-based Remote State Estimation

TL;DR

This paper introduces a deep reinforcement learning approach for dynamic radio resource allocation in NOMA-based remote state estimation, optimizing estimation accuracy over shared wireless channels with improved scalability and performance.

Contribution

It presents a novel RL-based resource allocation method for NOMA systems considering both channel and estimation states, addressing large hybrid action spaces.

Findings

The proposed algorithm effectively minimizes long-term estimation error.

It outperforms existing methods in scalability and performance.

Numerical results demonstrate significant gains over benchmarks.

Abstract

Remote state estimation, where many sensors send their measurements of distributed dynamic plants to a remote estimator over shared wireless resources, is essential for mission-critical applications of Industry 4.0. Most of the existing works on remote state estimation assumed orthogonal multiple access and the proposed dynamic radio resource allocation algorithms can only work for very small-scale settings. In this work, we consider a remote estimation system with non-orthogonal multiple access. We formulate a novel dynamic resource allocation problem for achieving the minimum overall long-term average estimation mean-square error. Both the estimation quality state and the channel quality state are taken into account for decision making at each time. The problem has a large hybrid discrete and continuous action space for joint channel assignment and power allocation. We propose a novel action-space compression method and develop an advanced deep reinforcement learning algorithm to solve the problem. Numerical results show that our algorithm solves the resource allocation problem effectively, presents much better scalability than the literature, and provides significant performance gain compared to some benchmarks.