UAV Trajectory Optimization via Improved Noisy Deep Q-Network

TL;DR

This paper introduces an improved Noisy Deep Q-Network for UAV trajectory optimization, enhancing exploration and stability in reinforcement learning, leading to faster convergence and higher rewards in simulated navigation tasks.

Contribution

The paper presents novel modifications to Noisy DQN, including residual NoisyLinear layers and adaptive noise scheduling, improving exploration and training stability for UAV applications.

Findings

Achieves up to 40% higher rewards than standard DQN.

Converges faster in grid navigation tasks.

Enhances exploration and stability in deep reinforcement learning.

Abstract

This paper proposes an Improved Noisy Deep Q-Network (Noisy DQN) to enhance the exploration and stability of Unmanned Aerial Vehicle (UAV) when applying deep reinforcement learning in simulated environments. This method enhances the exploration ability by combining the residual NoisyLinear layer with an adaptive noise scheduling mechanism, while improving training stability through smooth loss and soft target network updates. Experiments show that the proposed model achieves faster convergence and up to $+40$ higher rewards compared to standard DQN and quickly reach to the minimum number of steps required for the task 28 in the 15 * 15 grid navigation environment set up. The results show that our comprehensive improvements to the network structure of NoisyNet, exploration control, and training stability contribute to enhancing the efficiency and reliability of deep Q-learning.