PoseBench3D: A Cross-Dataset Analysis Framework for 3D Human Pose Estimation via Pose Lifting Networks

TL;DR

PoseBench3D is a standardized framework that automates cross-dataset evaluation of 3D human pose estimation methods, revealing generalization gaps and facilitating fair comparisons across diverse datasets.

Contribution

We introduce PoseBench3D, a comprehensive testing framework that enables automated, consistent cross-dataset evaluation of 3D human pose estimation methods.

Findings

Significant generalization gaps across datasets.

Preprocessing and dataset setup impact performance.

Re-evaluation of 18 methods under standardized protocols.

Abstract

Reliable three-dimensional human pose estimation (3D HPE) remains challenging due to the differences in viewpoints, environments, and camera conventions among datasets. As a result, methods that achieve near-optimal in-dataset accuracy often degrade on unseen datasets. In practice, however, systems must adapt to diverse viewpoints, environments, and camera setups--conditions that differ significantly from those encountered during training, which is often the case in real-world scenarios. Measuring cross-dataset performance is a vital process, but extremely labor-intensive when done manually for human pose estimation. To address these challenges, we automate this evaluation using PoseBench3D, a standardized testing framework that enables consistent and fair cross-dataset comparisons on previously unseen data. PoseBench3D streamlines testing across four widely used 3D HPE datasets via a single, configurable interface. Using this framework, we re-evaluate 18 methods and report over 100 cross-dataset results under Protocol 1: MPJPE and Protocol 2: PA-MPJPE, revealing systematic generalization gaps and the impact of common preprocessing and dataset setup choices. The PoseBench3D code is found at: https://github.com/bryanjvela/PoseBench3D