UPose3D: Uncertainty-Aware 3D Human Pose Estimation with Cross-View and Temporal Cues

TL;DR

UPose3D introduces an uncertainty-aware multi-view 3D human pose estimation method that leverages cross-view and temporal cues, improving robustness and scalability without requiring 3D annotations, achieving state-of-the-art results.

Contribution

The paper presents UPose3D, a novel framework that integrates uncertainty modeling with cross-view and temporal information for scalable 3D human pose estimation without direct 3D supervision.

Findings

State-of-the-art performance in out-of-distribution scenarios.

Competitive results with methods using 3D annotations.

Robustness to noisy and outlier data.

Abstract

We introduce UPose3D, a novel approach for multi-view 3D human pose estimation, addressing challenges in accuracy and scalability. Our method advances existing pose estimation frameworks by improving robustness and flexibility without requiring direct 3D annotations. At the core of our method, a pose compiler module refines predictions from a 2D keypoints estimator that operates on a single image by leveraging temporal and cross-view information. Our novel cross-view fusion strategy is scalable to any number of cameras, while our synthetic data generation strategy ensures generalization across diverse actors, scenes, and viewpoints. Finally, UPose3D leverages the prediction uncertainty of both the 2D keypoint estimator and the pose compiler module. This provides robustness to outliers and noisy data, resulting in state-of-the-art performance in out-of-distribution settings. In addition, for in-distribution settings, UPose3D yields performance rivalling methods that rely on 3D annotated data while being the state-of-the-art among methods relying only on 2D supervision.