The Event-Camera Dataset and Simulator: Event-based Data for Pose Estimation, Visual Odometry, and SLAM

TL;DR

This paper introduces a comprehensive dataset and simulator for event-based vision sensors, facilitating research in high-speed, high-dynamic-range robotics and computer vision tasks like pose estimation and SLAM.

Contribution

It provides a large, annotated dataset with real and synthetic event-camera data, inertial measurements, and ground-truth poses, along with an open-source simulator for synthetic data generation.

Findings

Dataset enables benchmarking of event-based algorithms.

Synthetic data helps develop and test new algorithms.

Ground-truth poses allow quantitative evaluation.

Abstract

New vision sensors, such as the Dynamic and Active-pixel Vision sensor (DAVIS), incorporate a conventional global-shutter camera and an event-based sensor in the same pixel array. These sensors have great potential for high-speed robotics and computer vision because they allow us to combine the benefits of conventional cameras with those of event-based sensors: low latency, high temporal resolution, and very high dynamic range. However, new algorithms are required to exploit the sensor characteristics and cope with its unconventional output, which consists of a stream of asynchronous brightness changes (called "events") and synchronous grayscale frames. For this purpose, we present and release a collection of datasets captured with a DAVIS in a variety of synthetic and real environments, which we hope will motivate research on new algorithms for high-speed and high-dynamic-range robotics and computer-vision applications. In addition to global-shutter intensity images and asynchronous events, we provide inertial measurements and ground-truth camera poses from a motion-capture system. The latter allows comparing the pose accuracy of ego-motion estimation algorithms quantitatively. All the data are released both as standard text files and binary files (i.e., rosbag). This paper provides an overview of the available data and describes a simulator that we release open-source to create synthetic event-camera data.