Event-Based high-speed low-latency fiducial marker tracking

TL;DR

This paper presents a real-time, low-latency fiducial marker tracking method using event-based sensors, achieving high-speed pose estimation suitable for fast robotic applications.

Contribution

It introduces an end-to-end event-based pipeline with a novel verification process for accurate 6-DOF pose estimation at unprecedented speeds.

Findings

Achieves pose estimation at up to 156 kHz.

Average latency of 3 ms.

Effective in fast robotic visual servoing.

Abstract

Motion and dynamic environments, especially under challenging lighting conditions, are still an open issue for robust robotic applications. In this paper, we propose an end-to-end pipeline for real-time, low latency, 6 degrees-of-freedom pose estimation of fiducial markers. Instead of achieving a pose estimation through a conventional frame-based approach, we employ the high-speed abilities of event-based sensors to directly refine the spatial transformation, using consecutive events. Furthermore, we introduce a novel two-way verification process for detecting tracking errors by backtracking the estimated pose, allowing us to evaluate the quality of our tracking. This approach allows us to achieve pose estimation at a rate up to 156~kHz, while only relying on CPU resources. The average end-to-end latency of our method is 3~ms. Experimental results demonstrate outstanding potential for robotic tasks, such as visual servoing in fast action-perception loops.