SMPLy Benchmarking 3D Human Pose Estimation in the Wild

TL;DR

This paper introduces a new benchmark dataset for 3D human pose estimation in the wild, enabling accurate evaluation of recent methods using a novel pipeline that leverages static scenes and online videos.

Contribution

It presents a cost-effective pipeline to generate and validate a large-scale in-the-wild dataset with ground-truth 3D poses for benchmarking.

Findings

State-of-the-art methods perform poorly on challenging poses.

Partial truncation and occlusion significantly affect accuracy.

Benchmark reveals remaining challenges in in-the-wild 3D pose estimation.

Abstract

Predicting 3D human pose from images has seen great recent improvements. Novel approaches that can even predict both pose and shape from a single input image have been introduced, often relying on a parametric model of the human body such as SMPL. While qualitative results for such methods are often shown for images captured in-the-wild, a proper benchmark in such conditions is still missing, as it is cumbersome to obtain ground-truth 3D poses elsewhere than in a motion capture room. This paper presents a pipeline to easily produce and validate such a dataset with accurate ground-truth, with which we benchmark recent 3D human pose estimation methods in-the-wild. We make use of the recently introduced Mannequin Challenge dataset which contains in-the-wild videos of people frozen in action like statues and leverage the fact that people are static and the camera moving to accurately fit the SMPL model on the sequences. A total of 24,428 frames with registered body models are then selected from 567 scenes at almost no cost, using only online RGB videos. We benchmark state-of-the-art SMPL-based human pose estimation methods on this dataset. Our results highlight that challenges remain, in particular for difficult poses or for scenes where the persons are partially truncated or occluded.