Leveraging 2D Masked Reconstruction for Domain Adaptation of 3D Pose Estimation

TL;DR

This paper introduces an unsupervised domain adaptation framework for 3D pose estimation using masked image modeling, improving accuracy across diverse datasets without requiring labeled data for new domains.

Contribution

It proposes a novel domain adaptation method leveraging masked image modeling and attention regularization to enhance 3D pose estimation across different data distributions.

Findings

Achieved state-of-the-art accuracy on multiple datasets.

Effective in cross-domain human and hand pose estimation.

Utilized unlabeled data to improve model robustness.

Abstract

RGB-based 3D pose estimation methods have been successful with the development of deep learning and the emergence of high-quality 3D pose datasets. However, most existing methods do not operate well for testing images whose distribution is far from that of training data. However, most existing methods do not operate well for testing images whose distribution is far from that of training data. This problem might be alleviated by involving diverse data during training, however it is non-trivial to collect such diverse data with corresponding labels (i.e. 3D pose). In this paper, we introduced an unsupervised domain adaptation framework for 3D pose estimation that utilizes the unlabeled data in addition to labeled data via masked image modeling (MIM) framework. Foreground-centric reconstruction and attention regularization are further proposed to increase the effectiveness of unlabeled data usage. Experiments are conducted on the various datasets in human and hand pose estimation tasks, especially using the cross-domain scenario. We demonstrated the effectiveness of ours by achieving the state-of-the-art accuracy on all datasets.