Motion Segmentation for Neuromorphic Aerial Surveillance

TL;DR

This paper introduces a self-supervised vision transformer-based motion segmentation method for event cameras, enabling accurate, scalable, and annotation-free detection of moving objects in aerial surveillance environments.

Contribution

The novel approach leverages self-supervised learning and vision transformers on event data and optical flow, eliminating the need for manual labels and scene-specific tuning.

Findings

Achieves state-of-the-art performance on multiple datasets.

Effectively handles various motion types and multiple objects.

Operates without human annotations or scene-specific parameters.

Abstract

Aerial surveillance demands rapid and precise detection of moving objects in dynamic environments. Event cameras, which draw inspiration from biological vision systems, present a promising alternative to frame-based sensors due to their exceptional temporal resolution, superior dynamic range, and minimal power requirements. Unlike traditional frame-based sensors that capture redundant information at fixed intervals, event cameras asynchronously record pixel-level brightness changes, providing a continuous and efficient data stream ideal for fast motion segmentation. While these sensors are ideal for fast motion segmentation, existing event-based motion segmentation methods often suffer from limitations such as the need for per-scene parameter tuning or reliance on manual labelling, hindering their scalability and practical deployment. In this paper, we address these challenges by introducing a novel motion segmentation method that leverages self-supervised vision transformers on both event data and optical flow information. Our approach eliminates the need for human annotations and reduces dependency on scene-specific parameters. In this paper, we used the EVK4-HD Prophesee event camera onboard a highly dynamic aerial platform in urban settings. We conduct extensive evaluations of our framework across multiple datasets, demonstrating state-of-the-art performance compared to existing benchmarks. Our method can effectively handle various types of motion and an arbitrary number of moving objects. Code and dataset are available at: \url{https://samiarja.github.io/evairborne/}