Learn by Observation: Imitation Learning for Drone Patrolling from Videos of A Human Navigator

TL;DR

This paper introduces an imitation learning approach for autonomous drone patrolling using only raw videos of human navigation, enabling high-altitude, broad-view, low-risk flight with accurate direction prediction.

Contribution

It proposes a novel imitation learning method that automatically collects and annotates data from videos, reducing manual effort and improving drone navigation accuracy.

Findings

High accuracy in predicting drone directions

Reliable autonomous navigation system demonstrated

Effective detection of accessible directions at crossroads

Abstract

We present an imitation learning method for autonomous drone patrolling based only on raw videos. Different from previous methods, we propose to let the drone learn patrolling in the air by observing and imitating how a human navigator does it on the ground. The observation process enables the automatic collection and annotation of data using inter-frame geometric consistency, resulting in less manual effort and high accuracy. Then a newly designed neural network is trained based on the annotated data to predict appropriate directions and translations for the drone to patrol in a lane-keeping manner as humans. Our method allows the drone to fly at a high altitude with a broad view and low risk. It can also detect all accessible directions at crossroads and further carry out the integration of available user instructions and autonomous patrolling control commands. Extensive experiments are conducted to demonstrate the accuracy of the proposed imitating learning process as well as the reliability of the holistic system for autonomous drone navigation. The codes, datasets as well as video demonstrations are available at https://vsislab.github.io/uavpatrol