CHAMP: Conformalized 3D Human Multi-Hypothesis Pose Estimators

TL;DR

CHAMP introduces a probabilistic framework for 3D human pose estimation from 2D keypoints, generating multiple hypotheses and using conformal prediction to select the most accurate ones, achieving state-of-the-art results.

Contribution

The paper presents a novel end-to-end differentiable conformal prediction method for multi-hypothesis 3D human pose estimation, improving accuracy and providing probabilistic guarantees.

Findings

State-of-the-art performance on multiple datasets.

Effective hypothesis scoring and aggregation improves accuracy.

Probabilistic guarantees enhance reliability of pose estimates.

Abstract

We introduce CHAMP, a novel method for learning sequence-to-sequence, multi-hypothesis 3D human poses from 2D keypoints by leveraging a conditional distribution with a diffusion model. To predict a single output 3D pose sequence, we generate and aggregate multiple 3D pose hypotheses. For better aggregation results, we develop a method to score these hypotheses during training, effectively integrating conformal prediction into the learning process. This process results in a differentiable conformal predictor that is trained end2end with the 3D pose estimator. Post-training, the learned scoring model is used as the conformity score, and the 3D pose estimator is combined with a conformal predictor to select the most accurate hypotheses for downstream aggregation. Our results indicate that using a simple mean aggregation on the conformal prediction-filtered hypotheses set yields competitive results. When integrated with more sophisticated aggregation techniques, our method achieves state-of-the-art performance across various metrics and datasets while inheriting the probabilistic guarantees of conformal prediction.