Towards End-to-End Neuromorphic Event-based 3D Object Reconstruction Without Physical Priors

TL;DR

This paper presents an end-to-end neuromorphic event-based 3D reconstruction method that enhances edge features without relying on physical priors, significantly improving accuracy in extreme environments.

Contribution

It introduces a novel event representation and an end-to-end model for dense voxel 3D reconstruction without physical priors, advancing the state-of-the-art in neuromorphic vision.

Findings

Achieved 54.6% improvement in reconstruction accuracy over baseline.

Developed a novel event representation to enhance edge features.

Proposed an optimal binarization threshold selection principle.

Abstract

Neuromorphic cameras, also known as event cameras, are asynchronous brightness-change sensors that can capture extremely fast motion without suffering from motion blur, making them particularly promising for 3D reconstruction in extreme environments. However, existing research on 3D reconstruction using monocular neuromorphic cameras is limited, and most of the methods rely on estimating physical priors and employ complex multi-step pipelines. In this work, we propose an end-to-end method for dense voxel 3D reconstruction using neuromorphic cameras that eliminates the need to estimate physical priors. Our method incorporates a novel event representation to enhance edge features, enabling the proposed feature-enhancement model to learn more effectively. Additionally, we introduced Optimal Binarization Threshold Selection Principle as a guideline for future related work, using the optimal reconstruction results achieved with threshold optimization as the benchmark. Our method achieves a 54.6% improvement in reconstruction accuracy compared to the baseline method.