DynamicPose: Real-time and Robust 6D Object Pose Tracking for Fast-Moving Cameras and Objects

TL;DR

DynamicPose is a real-time 6D object pose tracking framework that maintains robustness in fast-moving scenarios by integrating visual-inertial odometry, depth-informed 2D tracking, and a VIO-guided Kalman filter, without retraining.

Contribution

It introduces a retraining-free, closed-loop 6D pose tracking system combining multiple components to handle rapid camera and object movements effectively.

Findings

Achieves real-time 6D pose tracking in fast-moving scenarios.

Demonstrates robustness in both simulation and real-world tests.

Outperforms previous static-scene-based methods in dynamic environments.

Abstract

We present DynamicPose, a retraining-free 6D pose tracking framework that improves tracking robustness in fast-moving camera and object scenarios. Previous work is mainly applicable to static or quasi-static scenes, and its performance significantly deteriorates when both the object and the camera move rapidly. To overcome these challenges, we propose three synergistic components: (1) A visual-inertial odometry compensates for the shift in the Region of Interest (ROI) caused by camera motion; (2) A depth-informed 2D tracker corrects ROI deviations caused by large object translation; (3) A VIO-guided Kalman filter predicts object rotation, generates multiple candidate poses, and then obtains the final pose by hierarchical refinement. The 6D pose tracking results guide subsequent 2D tracking and Kalman filter updates, forming a closed-loop system that ensures accurate pose initialization and precise pose tracking. Simulation and real-world experiments demonstrate the effectiveness of our method, achieving real-time and robust 6D pose tracking for fast-moving cameras and objects.