0-MMS: Zero-Shot Multi-Motion Segmentation With A Monocular Event Camera

TL;DR

This paper introduces a novel monocular event camera-based method for zero-shot multi-motion segmentation in dynamic scenes, combining feature tracking and motion compensation, achieving state-of-the-art results and providing a new benchmark dataset.

Contribution

It presents the first unified pipeline for monocular multi-motion segmentation using event cameras, integrating bottom-up and top-down approaches, and introduces a new dataset for systematic evaluation.

Findings

Outperformed state-of-the-art detection rates by 12%

Achieved average detection rates of 81.06%, 94.2%, and 82.35% on three datasets

Proposed a real-time capable, unified segmentation approach for event cameras.

Abstract

Segmentation of moving objects in dynamic scenes is a key process in scene understanding for navigation tasks. Classical cameras suffer from motion blur in such scenarios rendering them effete. On the contrary, event cameras, because of their high temporal resolution and lack of motion blur, are tailor-made for this problem. We present an approach for monocular multi-motion segmentation, which combines bottom-up feature tracking and top-down motion compensation into a unified pipeline, which is the first of its kind to our knowledge. Using the events within a time-interval, our method segments the scene into multiple motions by splitting and merging. We further speed up our method by using the concept of motion propagation and cluster keyslices. The approach was successfully evaluated on both challenging real-world and synthetic scenarios from the EV-IMO, EED, and MOD datasets and outperformed the state-of-the-art detection rate by 12\%, achieving a new state-of-the-art average detection rate of 81.06%, 94.2% and 82.35% on the aforementioned datasets. To enable further research and systematic evaluation of multi-motion segmentation, we present and open-source a new dataset/benchmark called MOD++, which includes challenging sequences and extensive data stratification in-terms of camera and object motion, velocity magnitudes, direction, and rotational speeds.