Online Deep Reinforcement Learning for Autonomous UAV Navigation and Exploration of Outdoor Environments

TL;DR

This paper presents an Extended Double Deep Q-Network (EDDQN) approach for autonomous UAV navigation that leverages combined raw image and positional data to improve exploration and obstacle avoidance in outdoor environments under challenging weather conditions.

Contribution

The paper introduces a novel EDDQN method that integrates raw image and positional information for enhanced UAV navigation in complex, unseen outdoor environments.

Findings

EDDQN outperforms other deep Q-network variants

Effective navigation in unseen environments

Robust performance under severe weather conditions

Abstract

With the rapidly growing expansion in the use of UAVs, the ability to autonomously navigate in varying environments and weather conditions remains a highly desirable but as-of-yet unsolved challenge. In this work, we use Deep Reinforcement Learning to continuously improve the learning and understanding of a UAV agent while exploring a partially observable environment, which simulates the challenges faced in a real-life scenario. Our innovative approach uses a double state-input strategy that combines the acquired knowledge from the raw image and a map containing positional information. This positional data aids the network understanding of where the UAV has been and how far it is from the target position, while the feature map from the current scene highlights cluttered areas that are to be avoided. Our approach is extensively tested using variants of Deep Q-Network adapted to cope with double state input data. Further, we demonstrate that by altering the reward and the Q-value function, the agent is capable of consistently outperforming the adapted Deep Q-Network, Double Deep Q- Network and Deep Recurrent Q-Network. Our results demonstrate that our proposed Extended Double Deep Q-Network (EDDQN) approach is capable of navigating through multiple unseen environments and under severe weather conditions.