Distribution-Aware Coordinate Representation for Human Pose Estimation

TL;DR

This paper investigates the impact of coordinate representation and decoding methods in human pose estimation, proposing a novel distribution-aware approach that improves accuracy across multiple models and benchmarks.

Contribution

It introduces DARK, a distribution-aware coordinate representation method that enhances pose estimation performance by addressing decoding and encoding limitations.

Findings

DARK significantly improves accuracy on MPII and COCO benchmarks.

Decoding heatmaps into joint coordinates greatly affects pose estimation performance.

Distribution-aware encoding leads to more unbiased and effective training.

Abstract

While being the de facto standard coordinate representation in human pose estimation, heatmap is never systematically investigated in the literature, to our best knowledge. This work fills this gap by studying the coordinate representation with a particular focus on the heatmap. Interestingly, we found that the process of decoding the predicted heatmaps into the final joint coordinates in the original image space is surprisingly significant for human pose estimation performance, which nevertheless was not recognised before. In light of the discovered importance, we further probe the design limitations of the standard coordinate decoding method widely used by existing methods, and propose a more principled distribution-aware decoding method. Meanwhile, we improve the standard coordinate encoding process (i.e. transforming ground-truth coordinates to heatmaps) by generating accurate heatmap distributions for unbiased model training. Taking the two together, we formulate a novel Distribution-Aware coordinate Representation of Keypoint (DARK) method. Serving as a model-agnostic plug-in, DARK significantly improves the performance of a variety of state-of-the-art human pose estimation models. Extensive experiments show that DARK yields the best results on two common benchmarks, MPII and COCO, consistently validating the usefulness and effectiveness of our novel coordinate representation idea.