A Multimodal Learning-based Approach for Autonomous Landing of UAV

TL;DR

This paper presents a multimodal transformer-based detector and a reinforcement learning decision model to enhance autonomous UAV landing precision and resilience under environmental disturbances, validated in real outdoor scenarios.

Contribution

It introduces a novel multimodal transformer detector and a reinforcement learning-based decision model for UAV landing, improving reliability and adaptability in diverse conditions.

Findings

Achieved up to 90% true positive rate and precision.

Demonstrated rapid inference time of 5ms on edge devices.

Validated robustness in varying weather and sensor failure conditions.

Abstract

In the field of autonomous Unmanned Aerial Vehicles (UAVs) landing, conventional approaches fall short in delivering not only the required precision but also the resilience against environmental disturbances. Yet, learning-based algorithms can offer promising solutions by leveraging their ability to learn the intelligent behaviour from data. On one hand, this paper introduces a novel multimodal transformer-based Deep Learning detector, that can provide reliable positioning for precise autonomous landing. It surpasses standard approaches by addressing individual sensor limitations, achieving high reliability even in diverse weather and sensor failure conditions. It was rigorously validated across varying environments, achieving optimal true positive rates and average precisions of up to 90%. On the other hand, it is proposed a Reinforcement Learning (RL) decision-making model, based on a Deep Q-Network (DQN) rationale. Initially trained in sumlation, its adaptive behaviour is successfully transferred and validated in a real outdoor scenario. Furthermore, this approach demonstrates rapid inference times of approximately 5ms, validating its applicability on edge devices.