Optical Flow-Guided 6DoF Object Pose Tracking with an Event Camera

TL;DR

This paper introduces a novel event camera-based method for 6DoF object pose tracking that leverages optical flow and hybrid feature extraction to improve accuracy and robustness over existing approaches.

Contribution

It proposes a new optical flow-guided tracking framework using event cameras with a hybrid feature extraction strategy for precise pose estimation.

Findings

Outperforms state-of-the-art event-based methods in accuracy.

Demonstrates robustness under challenging conditions like motion blur and occlusion.

Effective in both simulated and real-world scenarios.

Abstract

Object pose tracking is one of the pivotal technologies in multimedia, attracting ever-growing attention in recent years. Existing methods employing traditional cameras encounter numerous challenges such as motion blur, sensor noise, partial occlusion, and changing lighting conditions. The emerging bio-inspired sensors, particularly event cameras, possess advantages such as high dynamic range and low latency, which hold the potential to address the aforementioned challenges. In this work, we present an optical flow-guided 6DoF object pose tracking method with an event camera. A 2D-3D hybrid feature extraction strategy is firstly utilized to detect corners and edges from events and object models, which characterizes object motion precisely. Then, we search for the optical flow of corners by maximizing the event-associated probability within a spatio-temporal window, and establish the correlation between corners and edges guided by optical flow. Furthermore, by minimizing the distances between corners and edges, the 6DoF object pose is iteratively optimized to achieve continuous pose tracking. Experimental results of both simulated and real events demonstrate that our methods outperform event-based state-of-the-art methods in terms of both accuracy and robustness.