Towards Deep Learning Assisted Autonomous UAVs for Manipulation Tasks in GPS-Denied Environments

TL;DR

This paper presents an integrated deep learning system enabling autonomous UAVs to perform complex object manipulation tasks in GPS-denied outdoor environments, inspired by MBZIRC 2020 challenge, with remote computing and novel hardware design.

Contribution

The paper introduces a novel multi-task visual perception system, grasp state estimation, and a remote computing approach for UAV manipulation in challenging environments.

Findings

Effective target localization and tracking in outdoor environments.

Successful deployment of autonomous UAV manipulation system.

High-speed wireless remote computing enhances performance.

Abstract

In this work, we present a pragmatic approach to enable unmanned aerial vehicle (UAVs) to autonomously perform highly complicated tasks of object pick and place. This paper is largely inspired by challenge-2 of MBZIRC 2020 and is primarily focused on the task of assembling large 3D structures in outdoors and GPS-denied environments. Primary contributions of this system are: (i) a novel computationally efficient deep learning based unified multi-task visual perception system for target localization, part segmentation, and tracking, (ii) a novel deep learning based grasp state estimation, (iii) a retracting electromagnetic gripper design, (iv) a remote computing approach which exploits state-of-the-art MIMO based high speed (5000Mb/s) wireless links to allow the UAVs to execute compute intensive tasks on remote high end compute servers, and (v) system integration in which several system components are weaved together in order to develop an optimized software stack. We use DJI Matrice-600 Pro, a hex-rotor UAV and interface it with the custom designed gripper. Our framework is deployed on the specified UAV in order to report the performance analysis of the individual modules. Apart from the manipulation system, we also highlight several hidden challenges associated with the UAVs in this context.