Dense Voxel 3D Reconstruction Using a Monocular Event Camera

TL;DR

This paper introduces a pioneering method for dense 3D reconstruction using only a single event camera, demonstrating promising results and providing a new synthetic dataset to facilitate further research.

Contribution

The paper presents the first approach for dense 3D reconstruction with a single event camera, bypassing complex pipelines used in prior multi-camera or semi-dense methods.

Findings

Produces visually distinguishable dense 3D reconstructions

Creates a synthetic dataset with nearly 40,000 object scans

Demonstrates feasibility of single event camera 3D reconstruction

Abstract

Event cameras are sensors inspired by biological systems that specialize in capturing changes in brightness. These emerging cameras offer many advantages over conventional frame-based cameras, including high dynamic range, high frame rates, and extremely low power consumption. Due to these advantages, event cameras have increasingly been adapted in various fields, such as frame interpolation, semantic segmentation, odometry, and SLAM. However, their application in 3D reconstruction for VR applications is underexplored. Previous methods in this field mainly focused on 3D reconstruction through depth map estimation. Methods that produce dense 3D reconstruction generally require multiple cameras, while methods that utilize a single event camera can only produce a semi-dense result. Other single-camera methods that can produce dense 3D reconstruction rely on creating a pipeline that either incorporates the aforementioned methods or other existing Structure from Motion (SfM) or Multi-view Stereo (MVS) methods. In this paper, we propose a novel approach for solving dense 3D reconstruction using only a single event camera. To the best of our knowledge, our work is the first attempt in this regard. Our preliminary results demonstrate that the proposed method can produce visually distinguishable dense 3D reconstructions directly without requiring pipelines like those used by existing methods. Additionally, we have created a synthetic dataset with $39,739$ object scans using an event camera simulator. This dataset will help accelerate other relevant research in this field.