The MONET dataset: Multimodal drone thermal dataset recorded in rural scenarios

TL;DR

The MONET dataset is a comprehensive multimodal thermal drone dataset capturing rural human and vehicle activities, designed to facilitate research in object localization and behavior understanding from moving viewpoints.

Contribution

This paper introduces MONET, a novel multimodal thermal drone dataset with extensive annotations, metadata, and diverse rural scenes, addressing gaps in existing datasets for large-scale and viewpoint variation.

Findings

Evaluated nine object detection algorithms on MONET.

Identified transfer learning challenges between different rural sites.

Highlighted open challenges in thermal drone data analysis.

Abstract

We present MONET, a new multimodal dataset captured using a thermal camera mounted on a drone that flew over rural areas, and recorded human and vehicle activities. We captured MONET to study the problem of object localisation and behaviour understanding of targets undergoing large-scale variations and being recorded from different and moving viewpoints. Target activities occur in two different land sites, each with unique scene structures and cluttered backgrounds. MONET consists of approximately 53K images featuring 162K manually annotated bounding boxes. Each image is timestamp-aligned with drone metadata that includes information about attitudes, speed, altitude, and GPS coordinates. MONET is different from previous thermal drone datasets because it features multimodal data, including rural scenes captured with thermal cameras containing both person and vehicle targets, along with trajectory information and metadata. We assessed the difficulty of the dataset in terms of transfer learning between the two sites and evaluated nine object detection algorithms to identify the open challenges associated with this type of data. Project page: https://github.com/fabiopoiesi/monet_dataset.