A Neuromorphic Dataset for Object Segmentation in Indoor Cluttered Environment

TL;DR

This paper introduces ESD, a comprehensive event-based dataset with high-quality annotations for object segmentation in cluttered indoor environments, addressing the lack of suitable benchmarks for event-based segmentation algorithms.

Contribution

The creation of the first densely annotated 3D spatial-temporal event-based dataset for indoor object segmentation using stereo event cameras.

Findings

ESD contains 145 sequences and over 14,000 RGB frames.

The dataset includes 21.88 million and 20.80 million events from stereo cameras.

ESD provides a challenging benchmark for segmentation in cluttered scenes.

Abstract

Taking advantage of an event-based camera, the issues of motion blur, low dynamic range and low time sampling of standard cameras can all be addressed. However, there is a lack of event-based datasets dedicated to the benchmarking of segmentation algorithms, especially those that provide depth information which is critical for segmentation in occluded scenes. This paper proposes a new Event-based Segmentation Dataset (ESD), a high-quality 3D spatial and temporal dataset for object segmentation in an indoor cluttered environment. Our proposed dataset ESD comprises 145 sequences with 14,166 RGB frames that are manually annotated with instance masks. Overall 21.88 million and 20.80 million events from two event-based cameras in a stereo-graphic configuration are collected, respectively. To the best of our knowledge, this densely annotated and 3D spatial-temporal event-based segmentation benchmark of tabletop objects is the first of its kind. By releasing ESD, we expect to provide the community with a challenging segmentation benchmark with high quality.