EV-IMO: Motion Segmentation Dataset and Learning Pipeline for Event Cameras

TL;DR

This paper introduces EV-IMO, the first event-based dataset and learning method for indoor motion segmentation, enabling accurate pixel-wise masks, egomotion, and object velocities using a low-parameter neural network.

Contribution

It provides the first event-based motion segmentation dataset and a novel efficient learning pipeline that estimates depth, egomotion, and object motion in indoor scenes.

Findings

Outperforms existing methods in motion segmentation accuracy

Handles fast motion and poor lighting effectively

Uses a shallow network with only 40k parameters

Abstract

We present the first event-based learning approach for motion segmentation in indoor scenes and the first event-based dataset - EV-IMO - which includes accurate pixel-wise motion masks, egomotion and ground truth depth. Our approach is based on an efficient implementation of the SfM learning pipeline using a low parameter neural network architecture on event data. In addition to camera egomotion and a dense depth map, the network estimates pixel-wise independently moving object segmentation and computes per-object 3D translational velocities for moving objects. We also train a shallow network with just 40k parameters, which is able to compute depth and egomotion. Our EV-IMO dataset features 32 minutes of indoor recording with up to 3 fast moving objects simultaneously in the camera field of view. The objects and the camera are tracked by the VICON motion capture system. By 3D scanning the room and the objects, accurate depth map ground truth and pixel-wise object masks are obtained, which are reliable even in poor lighting conditions and during fast motion. We then train and evaluate our learning pipeline on EV-IMO and demonstrate that our approach far surpasses its rivals and is well suited for scene constrained robotics applications.