Reinforcement Learning Based Power Control for Reliable Mission-Critical Wireless Transmission

TL;DR

This paper introduces a reinforcement learning framework for power control in mission-critical wireless transmissions, optimizing power usage while ensuring high success probability over fast-varying channels.

Contribution

It develops both model-based and model-free RL strategies for real-time power allocation, with the model-free approach achieving near-optimal performance.

Findings

The RL framework effectively solves the primal optimization problem.

The model-free strategy performs close to the model-based optimal algorithm.

The proposed methods improve power efficiency in mission-critical wireless links.

Abstract

In this paper, we investigate sequential power allocation over fast varying channels for mission-critical applications, aiming to minimize the expected sum power while guaranteeing the transmission success probability. In particular, a reinforcement learning framework is constructed with appropriate reward design so that the optimal policy maximizes the Lagrangian of the primal problem, where the maximizer of the Lagrangian is shown to have several good properties. For the model-based case, a fast converging algorithm is proposed to find the optimal Lagrange multiplier and thus the corresponding optimal policy. For the model-free case, we develop a three-stage strategy, composed in order of online sampling, offline learning, and online operation, where a backward Q-learning with full exploitation of sampled channel realizations is designed to accelerate the learning process. According to our simulation, the proposed reinforcement learning framework can solve the primal optimization problem from the dual perspective. Moreover, the model-free strategy achieves a performance close to that of the optimal model-based algorithm.