GoTrack: Generic 6DoF Object Pose Refinement and Tracking

TL;DR

GoTrack is a novel, efficient 6DoF object pose refinement and tracking method that combines model-to-frame and frame-to-frame registration using optical flow, achieving state-of-the-art results without object-specific training.

Contribution

It introduces a unified approach integrating analysis-by-synthesis and optical flow for robust, training-free object pose tracking applicable to diverse objects.

Findings

Achieves state-of-the-art RGB-only 6DoF pose estimation results.

Operates efficiently with simple neural network components.

Can be combined with existing coarse pose estimators for improved accuracy.

Abstract

We introduce GoTrack, an efficient and accurate CAD-based method for 6DoF object pose refinement and tracking, which can handle diverse objects without any object-specific training. Unlike existing tracking methods that rely solely on an analysis-by-synthesis approach for model-to-frame registration, GoTrack additionally integrates frame-to-frame registration, which saves compute and stabilizes tracking. Both types of registration are realized by optical flow estimation. The model-to-frame registration is noticeably simpler than in existing methods, relying only on standard neural network blocks (a transformer is trained on top of DINOv2) and producing reliable pose confidence scores without a scoring network. For the frame-to-frame registration, which is an easier problem as consecutive video frames are typically nearly identical, we employ a light off-the-shelf optical flow model. We demonstrate that GoTrack can be seamlessly combined with existing coarse pose estimation methods to create a minimal pipeline that reaches state-of-the-art RGB-only results on standard benchmarks for 6DoF object pose estimation and tracking. Our source code and trained models are publicly available at https://github.com/facebookresearch/gotrack