On the Calibration of Human Pose Estimation

TL;DR

This paper addresses the miscalibration of confidence scores in 2D human pose estimation, proposing a learning-based calibration method that improves accuracy metrics and downstream 3D mesh recovery.

Contribution

It introduces Calibrated ConfidenceNet (CCNet), a lightweight post-hoc calibration approach that aligns confidence with pose accuracy, enhancing existing pose estimation frameworks.

Findings

Improves AP by up to 1.4% on pose estimation benchmarks.

Reduces 3D keypoint error by 1.0mm in mesh recovery.

Provides theoretical analysis of calibration gaps in pose estimation.

Abstract

Most 2D human pose estimation frameworks estimate keypoint confidence in an ad-hoc manner, using heuristics such as the maximum value of heatmaps. The confidence is part of the evaluation scheme, e.g., AP for the MSCOCO dataset, yet has been largely overlooked in the development of state-of-the-art methods. This paper takes the first steps in addressing miscalibration in pose estimation. From a calibration point of view, the confidence should be aligned with the pose accuracy. In practice, existing methods are poorly calibrated. We show, through theoretical analysis, why a miscalibration gap exists and how to narrow the gap. Simply predicting the instance size and adjusting the confidence function gives considerable AP improvements. Given the black-box nature of deep neural networks, however, it is not possible to fully close this gap with only closed-form adjustments. As such, we go one step further and learn network-specific adjustments by enforcing consistency between confidence and pose accuracy. Our proposed Calibrated ConfidenceNet (CCNet) is a light-weight post-hoc addition that improves AP by up to 1.4% on off-the-shelf pose estimation frameworks. Applied to the downstream task of mesh recovery, CCNet facilitates an additional 1.0mm decrease in 3D keypoint error.