DensePose: Dense Human Pose Estimation In The Wild

TL;DR

This paper introduces DensePose, a system for dense human pose estimation in the wild, using a new dataset and CNN models to establish detailed correspondences between images and human body surfaces.

Contribution

It presents a large-scale dataset with dense annotations and a CNN-based approach for accurate, real-time dense human pose estimation in unconstrained environments.

Findings

Region-based models outperform fully-convolutional networks.

Inpainting network improves training effectiveness.

System achieves high accuracy in real-time applications.

Abstract

In this work, we establish dense correspondences between RGB image and a surface-based representation of the human body, a task we refer to as dense human pose estimation. We first gather dense correspondences for 50K persons appearing in the COCO dataset by introducing an efficient annotation pipeline. We then use our dataset to train CNN-based systems that deliver dense correspondence 'in the wild', namely in the presence of background, occlusions and scale variations. We improve our training set's effectiveness by training an 'inpainting' network that can fill in missing groundtruth values and report clear improvements with respect to the best results that would be achievable in the past. We experiment with fully-convolutional networks and region-based models and observe a superiority of the latter; we further improve accuracy through cascading, obtaining a system that delivers highly0accurate results in real time. Supplementary materials and videos are provided on the project page http://densepose.org