Learning Transferable Kinematic Dictionary for 3D Human Pose and Shape Reconstruction

TL;DR

This paper introduces a kinematic dictionary prior integrated with a neural network for 3D human pose and shape reconstruction from a single image, enabling end-to-end training and real-time performance.

Contribution

It proposes a novel kinematic dictionary prior that regularizes joint rotations, improving 3D reconstruction without shape annotations during training.

Findings

Achieves competitive results on Human3.6M, MPI-INF-3DHP, and LSP datasets.

Enables real-time 3D human pose and shape estimation.

Facilitates training across diverse datasets without shape annotations.

Abstract

Estimating 3D human pose and shape from a single image is highly under-constrained. To address this ambiguity, we propose a novel prior, namely kinematic dictionary, which explicitly regularizes the solution space of relative 3D rotations of human joints in the kinematic tree. Integrated with a statistical human model and a deep neural network, our method achieves end-to-end 3D reconstruction without the need of using any shape annotations during the training of neural networks. The kinematic dictionary bridges the gap between in-the-wild images and 3D datasets, and thus facilitates end-to-end training across all types of datasets. The proposed method achieves competitive results on large-scale datasets including Human3.6M, MPI-INF-3DHP, and LSP, while running in real-time given the human bounding boxes.