Delving into Robust Object Detection from Unmanned Aerial Vehicles: A Deep Nuisance Disentanglement Approach

TL;DR

This paper introduces a novel adversarial training framework called NDFT that leverages UAV meta-data to disentangle nuisances and significantly improve object detection robustness across diverse UAV imagery conditions.

Contribution

The paper proposes NDFT, a new adversarial learning approach that uses UAV meta-data to enhance object detection robustness against nuisances like altitude and weather.

Findings

Achieves state-of-the-art results on UAV detection benchmarks.

Significantly improves robustness to UAV-specific nuisances.

Demonstrates effectiveness of meta-data in domain adaptation.

Abstract

Object detection from images captured by Unmanned Aerial Vehicles (UAVs) is becoming increasingly useful. Despite the great success of the generic object detection methods trained on ground-to-ground images, a huge performance drop is observed when they are directly applied to images captured by UAVs. The unsatisfactory performance is owing to many UAV-specific nuisances, such as varying flying altitudes, adverse weather conditions, dynamically changing viewing angles, etc. Those nuisances constitute a large number of fine-grained domains, across which the detection model has to stay robust. Fortunately, UAVs will record meta-data that depict those varying attributes, which are either freely available along with the UAV images, or can be easily obtained. We propose to utilize those free meta-data in conjunction with associated UAV images to learn domain-robust features via an adversarial training framework dubbed Nuisance Disentangled Feature Transform (NDFT), for the specific challenging problem of object detection in UAV images, achieving a substantial gain in robustness to those nuisances. We demonstrate the effectiveness of our proposed algorithm, by showing state-of-the-art performance (single model) on two existing UAV-based object detection benchmarks. The code is available at https://github.com/TAMU-VITA/UAV-NDFT.