Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Image

TL;DR

This paper introduces SMPLify, a method that automatically estimates 3D human pose and shape from a single image by fitting a statistical body model to 2D joint detections, achieving high accuracy.

Contribution

It presents the first fully automatic approach to recover 3D human pose and shape from a single image using 2D joint detection and statistical modeling.

Findings

Outperforms previous methods in pose accuracy on benchmark datasets.

Uses minimal data for robust shape fitting due to population-based shape correlations.

Effectively handles occlusion, clothing, and lighting variations.

Abstract

We describe the first method to automatically estimate the 3D pose of the human body as well as its 3D shape from a single unconstrained image. We estimate a full 3D mesh and show that 2D joints alone carry a surprising amount of information about body shape. The problem is challenging because of the complexity of the human body, articulation, occlusion, clothing, lighting, and the inherent ambiguity in inferring 3D from 2D. To solve this, we first use a recently published CNN-based method, DeepCut, to predict (bottom-up) the 2D body joint locations. We then fit (top-down) a recently published statistical body shape model, called SMPL, to the 2D joints. We do so by minimizing an objective function that penalizes the error between the projected 3D model joints and detected 2D joints. Because SMPL captures correlations in human shape across the population, we are able to robustly fit it to very little data. We further leverage the 3D model to prevent solutions that cause interpenetration. We evaluate our method, SMPLify, on the Leeds Sports, HumanEva, and Human3.6M datasets, showing superior pose accuracy with respect to the state of the art.