Navigation in a Three-Dimensional Urban Flow using Deep Reinforcement Learning

TL;DR

This paper introduces a deep reinforcement learning approach for UAV navigation in complex 3D urban flows, leveraging flow-aware architectures to improve success and crash rates in turbulent environments.

Contribution

It presents a novel flow-aware PPO with GTrXL architecture for UAV navigation in turbulent urban flows, outperforming existing methods.

Findings

Significant increase in success rate compared to baseline algorithms.

Lower crash rate than traditional navigation methods.

Effective use of flow-aware deep reinforcement learning in turbulent environments.

Abstract

Unmanned Aerial Vehicles (UAVs) are increasingly populating urban areas for delivery and surveillance purposes. In this work, we develop an optimal navigation strategy based on Deep Reinforcement Learning. The environment is represented by a three-dimensional high-fidelity simulation of an urban flow, characterized by turbulence and recirculation zones. The algorithm presented here is a flow-aware Proximal Policy Optimization (PPO) combined with a Gated Transformer eXtra Large (GTrXL) architecture, giving the agent richer information about the turbulent flow field in which it navigates. The results are compared with a PPO+GTrXL without the secondary prediction tasks, a PPO combined with Long Short Term Memory (LSTM) cells and a traditional navigation algorithm. The obtained results show a significant increase in the success rate (SR) and a lower crash rate (CR) compared to a PPO+LSTM, PPO+GTrXL and the classical Zermelo's navigation algorithm, paving the way to a completely reimagined UAV landscape in complex urban environments.