SLAM-Supported Self-Training for 6D Object Pose Estimation

TL;DR

This paper introduces a SLAM-supported self-training approach that uses robot scene understanding and pose graph optimization to improve 6D object pose estimation accuracy in new environments.

Contribution

The method combines SLAM with self-training by integrating pose prediction covariances into pose graph optimization, automatically tuning uncertainties for better pseudo-label quality.

Findings

Achieves 34.3% accuracy improvement on YCB dataset

Achieves 17.8% accuracy improvement in real robot tests

Enhances 6D pose estimation robustness in novel environments

Abstract

Recent progress in object pose prediction provides a promising path for robots to build object-level scene representations during navigation. However, as we deploy a robot in novel environments, the out-of-distribution data can degrade the prediction performance. To mitigate the domain gap, we can potentially perform self-training in the target domain, using predictions on robot-captured images as pseudo labels to fine-tune the object pose estimator. Unfortunately, the pose predictions are typically outlier-corrupted, and it is hard to quantify their uncertainties, which can result in low-quality pseudo-labeled data. To address the problem, we propose a SLAM-supported self-training method, leveraging robot understanding of the 3D scene geometry to enhance the object pose inference performance. Combining the pose predictions with robot odometry, we formulate and solve pose graph optimization to refine the object pose estimates and make pseudo labels more consistent across frames. We incorporate the pose prediction covariances as variables into the optimization to automatically model their uncertainties. This automatic covariance tuning (ACT) process can fit 6D pose prediction noise at the component level, leading to higher-quality pseudo training data. We test our method with the deep object pose estimator (DOPE) on the YCB video dataset and in real robot experiments. It achieves respectively 34.3% and 17.8% accuracy enhancements in pose prediction on the two tests. Our code is available at https://github.com/520xyxyzq/slam-super-6d.