ES-PTAM: Event-based Stereo Parallel Tracking and Mapping

TL;DR

This paper introduces ES-PTAM, an event-based stereo visual odometry system that combines depth estimation and camera tracking, outperforming existing methods in various challenging scenarios with high-speed motion and dynamic lighting.

Contribution

The novel system integrates a correspondence-free depth estimation with edge-map based pose tracking, advancing event-based SLAM capabilities.

Findings

Outperforms state-of-the-art in multiple datasets

Reduces trajectory error significantly (up to 61%)

Works across diverse camera types and scenarios

Abstract

Visual Odometry (VO) and SLAM are fundamental components for spatial perception in mobile robots. Despite enormous progress in the field, current VO/SLAM systems are limited by their sensors' capability. Event cameras are novel visual sensors that offer advantages to overcome the limitations of standard cameras, enabling robots to expand their operating range to challenging scenarios, such as high-speed motion and high dynamic range illumination. We propose a novel event-based stereo VO system by combining two ideas: a correspondence-free mapping module that estimates depth by maximizing ray density fusion and a tracking module that estimates camera poses by maximizing edge-map alignment. We evaluate the system comprehensively on five real-world datasets, spanning a variety of camera types (manufacturers and spatial resolutions) and scenarios (driving, flying drone, hand-held, egocentric, etc). The quantitative and qualitative results demonstrate that our method outperforms the state of the art in majority of the test sequences by a margin, e.g., trajectory error reduction of 45% on RPG dataset, 61% on DSEC dataset, and 21% on TUM-VIE dataset. To benefit the community and foster research on event-based perception systems, we release the source code and results: https://github.com/tub-rip/ES-PTAM