A Survey on DRL based UAV Communications and Networking: DRL Fundamentals, Applications and Implementations

TL;DR

This survey reviews how deep reinforcement learning (DRL) techniques are applied to optimize UAV communication networks, addressing challenges like dynamic environments and large-scale scenarios, and highlights future research directions.

Contribution

It provides a comprehensive overview of DRL fundamentals, applications in UAV networks, and implementation details, emphasizing practical problem-solving in complex environments.

Findings

DRL effectively handles dynamic UAV network challenges

DRL algorithms improve optimization in large-scale scenarios

Future directions include multi-agent DRL applications

Abstract

Unmanned aerial vehicles (UAVs) are playing an increasingly pivotal role in modern communication networks,offering flexibility and enhanced coverage for a variety of applica-tions. However, UAV networks pose significant challenges due to their dynamic and distributed nature, particularly when dealing with tasks such as power allocation, channel assignment, caching,and task offloading. Traditional optimization techniques often struggle to handle the complexity and unpredictability of these environments, leading to suboptimal performance. This survey provides a comprehensive examination of how deep reinforcement learning (DRL) can be applied to solve these mathematical optimization problems in UAV communications and networking.Rather than simply introducing DRL methods, the focus is on demonstrating how these methods can be utilized to solve complex mathematical models of the underlying problems. We begin by reviewing the fundamental concepts of DRL, including value-based, policy-based, and actor-critic approaches. Then,we illustrate how DRL algorithms are applied to specific UAV network tasks by discussing from problem formulations to DRL implementation. By framing UAV communication challenges as optimization problems, this survey emphasizes the practical value of DRL in dynamic and uncertain environments. We also explore the strengths of DRL in handling large-scale network scenarios and the ability to continuously adapt to changes in the environment. In addition, future research directions are outlined, highlighting the potential for DRL to further enhance UAV communications and expand its applicability to more complex,multi-agent settings.