NU-AIR -- A Neuromorphic Urban Aerial Dataset for Detection and Localization of Pedestrians and Vehicles

TL;DR

NU-AIR is an open-source neuromorphic aerial dataset capturing urban pedestrians and vehicles, enabling research in event-based detection and localization with comprehensive annotations and validation.

Contribution

The paper introduces NU-AIR, a novel neuromorphic dataset for urban aerial perception, including annotations, validation, and open-source tools for SNN and DNN training.

Findings

Dataset contains 70.75 minutes of event footage with 93,204 labels.

Evaluation confirms dataset fidelity through ablation studies and neural network training.

Open-source code facilitates voxelization and model training.

Abstract

This paper presents an open-source aerial neuromorphic dataset that captures pedestrians and vehicles moving in an urban environment. The dataset, titled NU-AIR, features 70.75 minutes of event footage acquired with a 640 x 480 resolution neuromorphic sensor mounted on a quadrotor operating in an urban environment. Crowds of pedestrians, different types of vehicles, and street scenes featuring busy urban environments are captured at different elevations and illumination conditions. Manual bounding box annotations of vehicles and pedestrians contained in the recordings are provided at a frequency of 30 Hz, yielding 93,204 labels in total. Evaluation of the dataset's fidelity is performed through comprehensive ablation study for three Spiking Neural Networks (SNNs) and training ten Deep Neural Networks (DNNs) to validate the quality and reliability of both the dataset and corresponding annotations. All data and Python code to voxelize the data and subsequently train SNNs/DNNs has been open-sourced.