Event-based Motion & Appearance Fusion for 6D Object Pose Tracking

TL;DR

This paper introduces a novel event-camera-based method for 6D object pose tracking that leverages high temporal resolution and combines pose propagation with correction, outperforming some existing methods in fast-moving scenarios.

Contribution

It presents a learning-free approach that fuses event-based optical flow with pose correction, addressing limitations of traditional RGB-D sensors in dynamic environments.

Findings

Comparable performance to state-of-the-art algorithms

Outperforms in fast-moving object scenarios

Potential for high-speed dynamic environment applications

Abstract

Object pose tracking is a fundamental and essential task for robotics to perform tasks in the home and industrial settings. The most commonly used sensors to do so are RGB-D cameras, which can hit limitations in highly dynamic environments due to motion blur and frame-rate constraints. Event cameras have remarkable features such as high temporal resolution and low latency, which make them a potentially ideal vision sensors for object pose tracking at high speed. Even so, there are still only few works on 6D pose tracking with event cameras. In this work, we take advantage of the high temporal resolution and propose a method that uses both a propagation step fused with a pose correction strategy. Specifically, we use 6D object velocity obtained from event-based optical flow for pose propagation, after which, a template-based local pose correction module is utilized for pose correction. Our learning-free method has comparable performance to the state-of-the-art algorithms, and in some cases out performs them for fast-moving objects. The results indicate the potential for using event cameras in highly-dynamic scenarios where the use of deep network approaches are limited by low update rates.