Location-free Human Pose Estimation

TL;DR

This paper introduces a location-free human pose estimation framework that leverages weak supervision and a transformer-based model to achieve competitive accuracy with significantly fewer location annotations.

Contribution

It proposes a novel transformer-based approach that captures human context and structural relations without relying on keypoint annotations during training.

Findings

Achieves comparable results to fully-supervised methods with only 25% location labels.

Uses a multi-scale spatial-guided context encoder for global and part-aware features.

Employs a relation-encoded pose prototype module to model structural keypoint relations.

Abstract

Human pose estimation (HPE) usually requires large-scale training data to reach high performance. However, it is rather time-consuming to collect high-quality and fine-grained annotations for human body. To alleviate this issue, we revisit HPE and propose a location-free framework without supervision of keypoint locations. We reformulate the regression-based HPE from the perspective of classification. Inspired by the CAM-based weakly-supervised object localization, we observe that the coarse keypoint locations can be acquired through the part-aware CAMs but unsatisfactory due to the gap between the fine-grained HPE and the object-level localization. To this end, we propose a customized transformer framework to mine the fine-grained representation of human context, equipped with the structural relation to capture subtle differences among keypoints. Concretely, we design a Multi-scale Spatial-guided Context Encoder to fully capture the global human context while focusing on the part-aware regions and a Relation-encoded Pose Prototype Generation module to encode the structural relations. All these works together for strengthening the weak supervision from image-level category labels on locations. Our model achieves competitive performance on three datasets when only supervised at a category-level and importantly, it can achieve comparable results with fully-supervised methods with only 25\% location labels on MS-COCO and MPII.