CameraPose: Weakly-Supervised Monocular 3D Human Pose Estimation by Leveraging In-the-wild 2D Annotations

TL;DR

CameraPose is a weakly-supervised framework that leverages in-the-wild 2D annotations and a refinement network to improve monocular 3D human pose estimation, especially in unseen scenarios.

Contribution

It introduces a camera parameter branch and a confidence-guided refinement network, enabling training with 2D-only annotations and enhancing generalization to diverse poses.

Findings

Outperforms baseline by 3mm on 3DPW dataset

Improves existing 3D pose estimators in cross-scenario tests

Enhances training diversity with in-the-wild 2D annotations

Abstract

To improve the generalization of 3D human pose estimators, many existing deep learning based models focus on adding different augmentations to training poses. However, data augmentation techniques are limited to the "seen" pose combinations and hard to infer poses with rare "unseen" joint positions. To address this problem, we present CameraPose, a weakly-supervised framework for 3D human pose estimation from a single image, which can not only be applied on 2D-3D pose pairs but also on 2D alone annotations. By adding a camera parameter branch, any in-the-wild 2D annotations can be fed into our pipeline to boost the training diversity and the 3D poses can be implicitly learned by reprojecting back to 2D. Moreover, CameraPose introduces a refinement network module with confidence-guided loss to further improve the quality of noisy 2D keypoints extracted by 2D pose estimators. Experimental results demonstrate that the CameraPose brings in clear improvements on cross-scenario datasets. Notably, it outperforms the baseline method by 3mm on the most challenging dataset 3DPW. In addition, by combining our proposed refinement network module with existing 3D pose estimators, their performance can be improved in cross-scenario evaluation.