Semi-supervised 2D Human Pose Estimation via Adaptive Keypoint Masking

TL;DR

This paper introduces an adaptive keypoint masking technique for semi-supervised 2D human pose estimation, enhancing model performance by better leveraging unlabeled data and improving generalization on benchmark datasets.

Contribution

It proposes a novel adaptive keypoint masking method and a dual-branch data augmentation scheme to improve semi-supervised pose estimation accuracy and robustness.

Findings

Outperforms state-of-the-art on COCO by 5.2%

Achieves slight improvement on MPII by 0.3%

Enhances model generalization and robustness

Abstract

Human pose estimation is a fundamental and challenging task in computer vision. Larger-scale and more accurate keypoint annotations, while helpful for improving the accuracy of supervised pose estimation, are often expensive and difficult to obtain. Semi-supervised pose estimation tries to leverage a large amount of unlabeled data to improve model performance, which can alleviate the problem of insufficient labeled samples. The latest semi-supervised learning usually adopts a strong and weak data augmented teacher-student learning framework to deal with the challenge of "Human postural diversity and its long-tailed distribution". Appropriate data augmentation method is one of the key factors affecting the accuracy and generalization of semi-supervised models. Aiming at the problem that the difference of sample learning is not considered in the fixed keypoint masking augmentation method, this paper proposes an adaptive keypoint masking method, which can fully mine the information in the samples and obtain better estimation performance. In order to further improve the generalization and robustness of the model, this paper proposes a dual-branch data augmentation scheme, which can perform Mixup on samples and features on the basis of adaptive keypoint masking. The effectiveness of the proposed method is verified on COCO and MPII, outperforming the state-of-the-art semi-supervised pose estimation by 5.2% and 0.3%, respectively.