DSEC: A Stereo Event Camera Dataset for Driving Scenarios

TL;DR

DSEC is a comprehensive stereo event camera dataset designed for challenging driving scenarios with poor lighting, offering synchronized multi-sensor data including high-resolution event cameras, lidar, and GPS to advance autonomous driving research.

Contribution

This work introduces the first large-scale, high-resolution stereo dataset with event cameras for driving, covering diverse illumination conditions and providing ground truth disparity.

Findings

Includes 53 sequences in various lighting conditions

Provides synchronized data from stereo cameras, event cameras, lidar, and GPS

Facilitates development of event-based stereo algorithms

Abstract

Once an academic venture, autonomous driving has received unparalleled corporate funding in the last decade. Still, the operating conditions of current autonomous cars are mostly restricted to ideal scenarios. This means that driving in challenging illumination conditions such as night, sunrise, and sunset remains an open problem. In these cases, standard cameras are being pushed to their limits in terms of low light and high dynamic range performance. To address these challenges, we propose, DSEC, a new dataset that contains such demanding illumination conditions and provides a rich set of sensory data. DSEC offers data from a wide-baseline stereo setup of two color frame cameras and two high-resolution monochrome event cameras. In addition, we collect lidar data and RTK GPS measurements, both hardware synchronized with all camera data. One of the distinctive features of this dataset is the inclusion of high-resolution event cameras. Event cameras have received increasing attention for their high temporal resolution and high dynamic range performance. However, due to their novelty, event camera datasets in driving scenarios are rare. This work presents the first high-resolution, large-scale stereo dataset with event cameras. The dataset contains 53 sequences collected by driving in a variety of illumination conditions and provides ground truth disparity for the development and evaluation of event-based stereo algorithms.