Depth-CUPRL: Depth-Imaged Contrastive Unsupervised Prioritized Representations in Reinforcement Learning for Mapless Navigation of Unmanned Aerial Vehicles

TL;DR

This paper introduces Depth-CUPRL, a novel RL approach using depth map estimation and contrastive learning to improve UAV mapless navigation, outperforming pixel-based methods in decision-making efficiency.

Contribution

The paper presents a new method combining depth estimation, contrastive learning, and prioritized replay to enhance RL for UAV navigation without maps.

Findings

Depth-CUPRL outperforms state-of-the-art pixel-based methods.

The approach improves decision-making in UAV navigation.

Depth estimation enhances RL performance in high-dimensional observations.

Abstract

Reinforcement Learning (RL) has presented an impressive performance in video games through raw pixel imaging and continuous control tasks. However, RL performs poorly with high-dimensional observations such as raw pixel images. It is generally accepted that physical state-based RL policies such as laser sensor measurements give a more sample-efficient result than learning by pixels. This work presents a new approach that extracts information from a depth map estimation to teach an RL agent to perform the mapless navigation of Unmanned Aerial Vehicle (UAV). We propose the Depth-Imaged Contrastive Unsupervised Prioritized Representations in Reinforcement Learning(Depth-CUPRL) that estimates the depth of images with a prioritized replay memory. We used a combination of RL and Contrastive Learning to lead with the problem of RL based on images. From the analysis of the results with Unmanned Aerial Vehicles (UAVs), it is possible to conclude that our Depth-CUPRL approach is effective for the decision-making and outperforms state-of-the-art pixel-based approaches in the mapless navigation capability.