Learning Monocular Dense Depth from Events

TL;DR

This paper introduces a recurrent neural network architecture for monocular dense depth prediction from event camera data, demonstrating significant accuracy improvements over previous methods.

Contribution

It presents the first recurrent approach for dense depth prediction from event streams and introduces a new pretraining dataset using CARLA simulator data.

Findings

Up to 50% reduction in average depth error compared to prior methods

Recurrent architecture leverages temporal consistency in event streams

Effective monocular dense depth prediction from event data

Abstract

Event cameras are novel sensors that output brightness changes in the form of a stream of asynchronous events instead of intensity frames. Compared to conventional image sensors, they offer significant advantages: high temporal resolution, high dynamic range, no motion blur, and much lower bandwidth. Recently, learning-based approaches have been applied to event-based data, thus unlocking their potential and making significant progress in a variety of tasks, such as monocular depth prediction. Most existing approaches use standard feed-forward architectures to generate network predictions, which do not leverage the temporal consistency presents in the event stream. We propose a recurrent architecture to solve this task and show significant improvement over standard feed-forward methods. In particular, our method generates dense depth predictions using a monocular setup, which has not been shown previously. We pretrain our model using a new dataset containing events and depth maps recorded in the CARLA simulator. We test our method on the Multi Vehicle Stereo Event Camera Dataset (MVSEC). Quantitative experiments show up to 50% improvement in average depth error with respect to previous event-based methods.