IndraEye: Infrared Electro-Optical UAV-based Perception Dataset for Robust Downstream Tasks

TL;DR

IndraEye is a comprehensive multi-sensor UAV dataset designed to improve the robustness of deep learning models for aerial perception across various challenging conditions, including low-light and diverse viewing angles.

Contribution

The paper introduces the IndraEye dataset, a large-scale multi-sensor UAV dataset with diverse conditions, enabling research in multimodal learning, domain adaptation, and sensor analysis for aerial perception.

Findings

Benchmark results for object detection and segmentation on IndraEye

Dataset covers multiple viewing angles, altitudes, and lighting conditions

Supports development of robust UAV perception systems

Abstract

Deep neural networks (DNNs) have shown exceptional performance when trained on well-illuminated images captured by Electro-Optical (EO) cameras, which provide rich texture details. However, in critical applications like aerial perception, it is essential for DNNs to maintain consistent reliability across all conditions, including low-light scenarios where EO cameras often struggle to capture sufficient detail. Additionally, UAV-based aerial object detection faces significant challenges due to scale variability from varying altitudes and slant angles, adding another layer of complexity. Existing methods typically address only illumination changes or style variations as domain shifts, but in aerial perception, correlation shifts also impact DNN performance. In this paper, we introduce the IndraEye dataset, a multi-sensor (EO-IR) dataset designed for various tasks. It includes 5,612 images with 145,666 instances, encompassing multiple viewing angles, altitudes, seven backgrounds, and different times of the day across the Indian subcontinent. The dataset opens up several research opportunities, such as multimodal learning, domain adaptation for object detection and segmentation, and exploration of sensor-specific strengths and weaknesses. IndraEye aims to advance the field by supporting the development of more robust and accurate aerial perception systems, particularly in challenging conditions. IndraEye dataset is benchmarked with object detection and semantic segmentation tasks. Dataset and source codes are available at https://bit.ly/indraeye.