Joint Cluster Head Selection and Trajectory Planning in UAV-Aided IoT Networks by Reinforcement Learning with Sequential Model

TL;DR

This paper introduces a deep reinforcement learning approach with a sequential model to jointly optimize UAV trajectory and cluster head selection in IoT networks, significantly reducing energy consumption and generalizing well to larger problems.

Contribution

It presents a novel DRL method with a sequence-to-sequence neural network for energy-efficient UAV trajectory and cluster head optimization in IoT networks.

Findings

DRL method reduces UAV energy consumption compared to baselines.

The approach achieves near-optimal performance in simulations.

The trained model generalizes well to larger problem sizes.

Abstract

Employing unmanned aerial vehicles (UAVs) has attracted growing interests and emerged as the state-of-the-art technology for data collection in Internet-of-Things (IoT) networks. In this paper, with the objective of minimizing the total energy consumption of the UAV-IoT system, we formulate the problem of jointly designing the UAV's trajectory and selecting cluster heads in the IoT network as a constrained combinatorial optimization problem which is classified as NP-hard and challenging to solve. We propose a novel deep reinforcement learning (DRL) with a sequential model strategy that can effectively learn the policy represented by a sequence-to-sequence neural network for the UAV's trajectory design in an unsupervised manner. Through extensive simulations, the obtained results show that the proposed DRL method can find the UAV's trajectory that requires much less energy consumption when compared to other baseline algorithms and achieves close-to-optimal performance. In addition, simulation results show that the trained model by our proposed DRL algorithm has an excellent generalization ability to larger problem sizes without the need to retrain the model.