View Consistency Aware Holistic Triangulation for 3D Human Pose Estimation

TL;DR

This paper presents a novel multi-view fusion and holistic triangulation approach with anatomy priors for 3D human pose estimation, improving accuracy and plausibility by addressing view inconsistency and pose coherence.

Contribution

It introduces a multi-view fusion module and a holistic triangulation method with anatomy priors, enabling end-to-end training and superior 3D pose estimation performance.

Findings

Outperforms state-of-the-art in accuracy and plausibility

Introduces a new metric for pose plausibility

Effective integration of anatomy prior via PCA

Abstract

The rapid development of multi-view 3D human pose estimation (HPE) is attributed to the maturation of monocular 2D HPE and the geometry of 3D reconstruction. However, 2D detection outliers in occluded views due to neglect of view consistency, and 3D implausible poses due to lack of pose coherence, remain challenges. To solve this, we introduce a Multi-View Fusion module to refine 2D results by establishing view correlations. Then, Holistic Triangulation is proposed to infer the whole pose as an entirety, and anatomy prior is injected to maintain the pose coherence and improve the plausibility. Anatomy prior is extracted by PCA whose input is skeletal structure features, which can factor out global context and joint-by-joint relationship from abstract to concrete. Benefiting from the closed-form solution, the whole framework is trained end-to-end. Our method outperforms the state of the art in both precision and plausibility which is assessed by a new metric.