ProbPose: A Probabilistic Approach to 2D Human Pose Estimation

TL;DR

ProbPose introduces a probabilistic framework for 2D human pose estimation that effectively handles out-of-image keypoints and improves localization accuracy, especially at image boundaries, through calibrated probability predictions and new evaluation protocols.

Contribution

It proposes ProbPose, a novel probabilistic model for keypoint detection that accounts for out-of-image points and introduces the CropCOCO dataset and Ex-OKS metric for better evaluation.

Findings

Significant improvement in out-of-image keypoint localization.

Enhanced robustness at image edges.

Better flexibility in keypoint evaluation.

Abstract

Current Human Pose Estimation methods have achieved significant improvements. However, state-of-the-art models ignore out-of-image keypoints and use uncalibrated heatmaps as keypoint location representations. To address these limitations, we propose ProbPose, which predicts for each keypoint: a calibrated probability of keypoint presence at each location in the activation window, the probability of being outside of it, and its predicted visibility. To address the lack of evaluation protocols for out-of-image keypoints, we introduce the CropCOCO dataset and the Extended OKS (Ex-OKS) metric, which extends OKS to out-of-image points. Tested on COCO, CropCOCO, and OCHuman, ProbPose shows significant gains in out-of-image keypoint localization while also improving in-image localization through data augmentation. Additionally, the model improves robustness along the edges of the bounding box and offers better flexibility in keypoint evaluation. The code and models are available on https://mirapurkrabek.github.io/ProbPose/ for research purposes.