Probabilistic Triangulation for Uncalibrated Multi-View 3D Human Pose Estimation

TL;DR

This paper introduces a probabilistic triangulation approach that enables uncalibrated multi-view 3D human pose estimation, improving generalization and maintaining high accuracy through iterative Bayesian updates and end-to-end training.

Contribution

It proposes a novel probabilistic triangulation module that models camera pose as a distribution and updates it iteratively, allowing uncalibrated multi-view 3D pose estimation with end-to-end learning.

Findings

Outperforms other uncalibration methods on Human3.6M and CMU Panoptic datasets.

Achieves comparable results to calibration-based methods.

Enables end-to-end training with back-propagation from 3D to 2D features.

Abstract

3D human pose estimation has been a long-standing challenge in computer vision and graphics, where multi-view methods have significantly progressed but are limited by the tedious calibration processes. Existing multi-view methods are restricted to fixed camera pose and therefore lack generalization ability. This paper presents a novel Probabilistic Triangulation module that can be embedded in a calibrated 3D human pose estimation method, generalizing it to uncalibration scenes. The key idea is to use a probability distribution to model the camera pose and iteratively update the distribution from 2D features instead of using camera pose. Specifically, We maintain a camera pose distribution and then iteratively update this distribution by computing the posterior probability of the camera pose through Monte Carlo sampling. This way, the gradients can be directly back-propagated from the 3D pose estimation to the 2D heatmap, enabling end-to-end training. Extensive experiments on Human3.6M and CMU Panoptic demonstrate that our method outperforms other uncalibration methods and achieves comparable results with state-of-the-art calibration methods. Thus, our method achieves a trade-off between estimation accuracy and generalizability. Our code is in https://github.com/bymaths/probabilistic_triangulation