MSDA: Monocular Self-supervised Domain Adaptation for 6D Object Pose Estimation

TL;DR

This paper introduces a self-supervised domain adaptation method for 6D object pose estimation that leverages real RGB(-D) data without requiring pose labels, improving performance on real images using synthetic training.

Contribution

It presents a novel self-supervised fine-tuning approach that reduces the synthetic-to-real domain gap without needing real pose annotations, enhancing existing pose estimators.

Findings

Achieves comparable performance to fully-supervised methods.

Outperforms existing state-of-the-art approaches.

Effective use of real RGB(-D) data without pose labels.

Abstract

Acquiring labeled 6D poses from real images is an expensive and time-consuming task. Though massive amounts of synthetic RGB images are easy to obtain, the models trained on them suffer from noticeable performance degradation due to the synthetic-to-real domain gap. To mitigate this degradation, we propose a practical self-supervised domain adaptation approach that takes advantage of real RGB(-D) data without needing real pose labels. We first pre-train the model with synthetic RGB images and then utilize real RGB(-D) images to fine-tune the pre-trained model. The fine-tuning process is self-supervised by the RGB-based pose-aware consistency and the depth-guided object distance pseudo-label, which does not require the time-consuming online differentiable rendering. We build our domain adaptation method based on the recent pose estimator SC6D and evaluate it on the YCB-Video dataset. We experimentally demonstrate that our method achieves comparable performance against its fully-supervised counterpart while outperforming existing state-of-the-art approaches.