Kinematic-Structure-Preserved Representation for Unsupervised 3D Human Pose Estimation

TL;DR

This paper introduces a novel unsupervised 3D human pose estimation method that preserves kinematic structure, enabling effective learning from in-the-wild videos without paired supervision, and demonstrates state-of-the-art results and strong generalization.

Contribution

It proposes a kinematic-structure-preserved unsupervised framework that relies on skeletal priors and differentiable transformations, avoiding adversarial training and paired supervision.

Findings

Achieves state-of-the-art unsupervised pose estimation on benchmark datasets.

Demonstrates superior generalization to unseen environments.

Operates effectively without paired or weak supervision.

Abstract

Estimation of 3D human pose from monocular image has gained considerable attention, as a key step to several human-centric applications. However, generalizability of human pose estimation models developed using supervision on large-scale in-studio datasets remains questionable, as these models often perform unsatisfactorily on unseen in-the-wild environments. Though weakly-supervised models have been proposed to address this shortcoming, performance of such models relies on availability of paired supervision on some related tasks, such as 2D pose or multi-view image pairs. In contrast, we propose a novel kinematic-structure-preserved unsupervised 3D pose estimation framework, which is not restrained by any paired or unpaired weak supervisions. Our pose estimation framework relies on a minimal set of prior knowledge that defines the underlying kinematic 3D structure, such as skeletal joint connectivity information with bone-length ratios in a fixed canonical scale. The proposed model employs three consecutive differentiable transformations named as forward-kinematics, camera-projection and spatial-map transformation. This design not only acts as a suitable bottleneck stimulating effective pose disentanglement but also yields interpretable latent pose representations avoiding training of an explicit latent embedding to pose mapper. Furthermore, devoid of unstable adversarial setup, we re-utilize the decoder to formalize an energy-based loss, which enables us to learn from in-the-wild videos, beyond laboratory settings. Comprehensive experiments demonstrate our state-of-the-art unsupervised and weakly-supervised pose estimation performance on both Human3.6M and MPI-INF-3DHP datasets. Qualitative results on unseen environments further establish our superior generalization ability.