Estimating Human Poses Across Datasets: A Unified Skeleton and Multi-Teacher Distillation Approach

TL;DR

This paper introduces a unified skeleton representation and multi-teacher knowledge distillation to improve human pose estimation across different datasets, enhancing model generalization and accuracy in predicting extended keypoints.

Contribution

It presents a novel method combining multi-teacher distillation with a unified skeleton to address dataset annotation inconsistencies in human pose estimation.

Findings

Achieved higher cross-dataset accuracy (70.89 and 76.40) compared to single dataset training.

Predicted 21 keypoints with AP of 66.84 and 72.75 on Halpe dataset.

Enhanced model adaptability to different datasets and annotations.

Abstract

Human pose estimation is a key task in computer vision with various applications such as activity recognition and interactive systems. However, the lack of consistency in the annotated skeletons across different datasets poses challenges in developing universally applicable models. To address this challenge, we propose a novel approach integrating multi-teacher knowledge distillation with a unified skeleton representation. Our networks are jointly trained on the COCO and MPII datasets, containing 17 and 16 keypoints, respectively. We demonstrate enhanced adaptability by predicting an extended set of 21 keypoints, 4 (COCO) and 5 (MPII) more than original annotations, improving cross-dataset generalization. Our joint models achieved an average accuracy of 70.89 and 76.40, compared to 53.79 and 55.78 when trained on a single dataset and evaluated on both. Moreover, we also evaluate all 21 predicted points by our two models by reporting an AP of 66.84 and 72.75 on the Halpe dataset. This highlights the potential of our technique to address one of the most pressing challenges in pose estimation research and application - the inconsistency in skeletal annotations.