HPRNet: Hierarchical Point Regression for Whole-Body Human Pose Estimation

TL;DR

HPRNet introduces a hierarchical point regression approach for whole-body human pose estimation, effectively handling scale variance across body parts and achieving state-of-the-art results in keypoint detection.

Contribution

The paper proposes a novel bottom-up, one-stage hierarchical point regression method for whole-body pose estimation, addressing scale variance and enabling constant-time inference regardless of the number of people.

Findings

Outperforms previous bottom-up methods on COCO WholeBody dataset

Achieves state-of-the-art face keypoint detection (75.4 AP)

Achieves state-of-the-art hand keypoint detection (50.4 AP)

Abstract

In this paper, we present a new bottom-up one-stage method for whole-body pose estimation, which we call "hierarchical point regression," or HPRNet for short. In standard body pose estimation, the locations of $\sim 17$ major joints on the human body are estimated. Differently, in whole-body pose estimation, the locations of fine-grained keypoints (68 on face, 21 on each hand and 3 on each foot) are estimated as well, which creates a scale variance problem that needs to be addressed. To handle the scale variance among different body parts, we build a hierarchical point representation of body parts and jointly regress them. The relative locations of fine-grained keypoints in each part (e.g. face) are regressed in reference to the center of that part, whose location itself is estimated relative to the person center. In addition, unlike the existing two-stage methods, our method predicts whole-body pose in a constant time independent of the number of people in an image. On the COCO WholeBody dataset, HPRNet significantly outperforms all previous bottom-up methods on the keypoint detection of all whole-body parts (i.e. body, foot, face and hand); it also achieves state-of-the-art results on face (75.4 AP) and hand (50.4 AP) keypoint detection. Code and models are available at \url{https://github.com/nerminsamet/HPRNet}.