Distributional Reinforcement Learning for mmWave Communications with Intelligent Reflectors on a UAV

TL;DR

This paper introduces a distributional reinforcement learning framework to optimize UAV-mounted intelligent reflectors for enhancing mmWave communication capacity and reliability in multi-user downlink scenarios.

Contribution

It proposes a novel joint optimization of precoding and reflection coefficients, and a distributional RL method for UAV-IR placement to improve mmWave communication performance.

Findings

Significant increase in average data rate with RL-based UAV-IR deployment.

Higher line-of-sight probability achieved through the proposed method.

Outperforms static IR and direct transmission schemes in simulations.

Abstract

In this paper, a novel communication framework that uses an unmanned aerial vehicle (UAV)-carried intelligent reflector (IR) is proposed to enhance multi-user downlink transmissions over millimeter wave (mmWave) frequencies. In order to maximize the downlink sum-rate, the optimal precoding matrix (at the base station) and reflection coefficient (at the IR) are jointly derived. Next, to address the uncertainty of mmWave channels and maintain line-of-sight links in a real-time manner, a distributional reinforcement learning approach, based on quantile regression optimization, is proposed to learn the propagation environment of mmWave communications, and, then, optimize the location of the UAV-IR so as to maximize the long-term downlink communication capacity. Simulation results show that the proposed learning-based deployment of the UAV-IR yields a significant advantage, compared to a non-learning UAV-IR, a static IR, and a direct transmission schemes, in terms of the average data rate and the achievable line-of-sight probability of downlink mmWave communications.