DeepSkeleton: Skeleton Map for 3D Human Pose Regression

TL;DR

This paper introduces Skeleton Map, an intermediate representation for 3D human pose regression that effectively captures structural context, enabling competitive performance with state-of-the-art methods using only skeleton maps and multiple hypotheses.

Contribution

It proposes Skeleton Map as a novel intermediate feature for 3D pose estimation, simplifying the regression process and improving accuracy without relying on RGB image details.

Findings

Skeleton Map alone achieves state-of-the-art performance.

Multiple 3D hypotheses improve pose estimation consistency.

Effective on both in-the-wild and indoor datasets.

Abstract

Despite recent success on 2D human pose estimation, 3D human pose estimation still remains an open problem. A key challenge is the ill-posed depth ambiguity nature. This paper presents a novel intermediate feature representation named skeleton map for regression. It distills structural context from irrelavant properties of RGB image e.g. illumination and texture. It is simple, clean and can be easily generated via deconvolution network. For the first time, we show that training regression network from skeleton map alone is capable of meeting the performance of state-of-theart 3D human pose estimation works. We further exploit the power of multiple 3D hypothesis generation to obtain reasonbale 3D pose in consistent with 2D pose detection. The effectiveness of our approach is validated on challenging in-the-wild dataset MPII and indoor dataset Human3.6M.