DQN-based Beamforming for Uplink mmWave Cellular-Connected UAVs

TL;DR

This paper introduces a reinforcement learning framework using deep Q-Networks for efficient beam alignment in UAV mmWave communications, significantly reducing overhead and adapting to dynamic environments.

Contribution

It presents a novel DQN-based method for UAV beam alignment that outperforms traditional approaches in speed and adaptability, suitable for real-time deployment.

Findings

DQN-based approach converges faster than MAB and exhaustive methods.

The framework adapts well to different environmental conditions.

Achieves near-optimal beam alignment in real-time scenarios.

Abstract

Unmanned aerial vehicles (UAVs) are the emerging vital components of millimeter wave (mmWave) wireless systems. Accurate beam alignment is essential for efficient beam-based mmWave communications of UAVs with base stations (BSs). Conventional beam sweeping approaches often have large overhead due to the high mobility and autonomous operation of UAVs. Learning-based approaches greatly reduce the overhead by leveraging UAV data, like position to identify optimal beam directions. In this paper, we propose a reinforcement learning (RL)-based framework for UAV-BS beam alignment using deep Q-Network (DQN) in a mmWave setting. We consider uplink communications where the UAV hovers around 5G new radio (NR) BS coverage area, with varying channel conditions. The proposed learning framework uses the location information to maximize data rate through the optimal beam-pairs efficiently, upon every communication request from UAV inside the multi-location environment. We compare our proposed framework against the Multi-Armed Bandit (MAB) learning-based approach and the traditional exhaustive approach, respectively, and also analyse the training performance of DQN-based beam alignment over different coverage area requirements and channel conditions. Our results show that the proposed DQN-based beam alignment converges faster and generic for different environmental conditions. The framework can also learn optimal beam alignment comparable to the exhaustive approach in an online manner under real-time conditions.