Visual Odometry with an Event Camera Using Continuous Ray Warping and Volumetric Contrast Maximization

TL;DR

This paper introduces a novel 3D contrast maximization approach for event camera-based visual odometry, enabling joint motion and structure estimation in complex environments, outperforming traditional methods especially in challenging conditions.

Contribution

The paper proposes a continuous ray warping and volumetric contrast maximization technique for event cameras, addressing the limitations of 2D image warping in complex scenes.

Findings

Approaches regular camera performance in motion estimation accuracy.

Outperforms existing event-based methods in challenging visual conditions.

Enables joint optimization of motion and 3D structure from event data.

Abstract

We present a new solution to tracking and mapping with an event camera. The motion of the camera contains both rotation and translation, and the displacements happen in an arbitrarily structured environment. As a result, the image matching may no longer be represented by a low-dimensional homographic warping, thus complicating an application of the commonly used Image of Warped Events (IWE). We introduce a new solution to this problem by performing contrast maximization in 3D. The 3D location of the rays cast for each event is smoothly varied as a function of a continuous-time motion parametrization, and the optimal parameters are found by maximizing the contrast in a volumetric ray density field. Our method thus performs joint optimization over motion and structure. The practical validity of our approach is supported by an application to AGV motion estimation and 3D reconstruction with a single vehicle-mounted event camera. The method approaches the performance obtained with regular cameras, and eventually outperforms in challenging visual conditions.