HDNet: Human Depth Estimation for Multi-Person Camera-Space Localization

TL;DR

HDNet is an end-to-end framework that accurately estimates the absolute depth of human root joints in multi-person scenes, improving 3D pose localization in camera space.

Contribution

The paper introduces HDNet, a novel deep learning model that combines 2D pose estimation, attention mechanisms, and GNNs for absolute root joint localization in 3D space.

Findings

Outperforms previous state-of-the-art on Human3.6M and MuPoTS-3D datasets.

Achieves higher accuracy in root joint localization and 3D pose estimation.

Demonstrates robustness across multiple evaluation metrics.

Abstract

Current works on multi-person 3D pose estimation mainly focus on the estimation of the 3D joint locations relative to the root joint and ignore the absolute locations of each pose. In this paper, we propose the Human Depth Estimation Network (HDNet), an end-to-end framework for absolute root joint localization in the camera coordinate space. Our HDNet first estimates the 2D human pose with heatmaps of the joints. These estimated heatmaps serve as attention masks for pooling features from image regions corresponding to the target person. A skeleton-based Graph Neural Network (GNN) is utilized to propagate features among joints. We formulate the target depth regression as a bin index estimation problem, which can be transformed with a soft-argmax operation from the classification output of our HDNet. We evaluate our HDNet on the root joint localization and root-relative 3D pose estimation tasks with two benchmark datasets, i.e., Human3.6M and MuPoTS-3D. The experimental results show that we outperform the previous state-of-the-art consistently under multiple evaluation metrics. Our source code is available at: https://github.com/jiahaoLjh/HumanDepth.