VolNet: Estimating Human Body Part Volumes from a Single RGB Image

TL;DR

VolNet is a novel deep learning architecture that estimates human body volume from a single RGB image by combining pose estimation, segmentation, and volume regression, significantly outperforming previous methods.

Contribution

The paper introduces VolNet, a new model leveraging 2D/3D pose, segmentation, and volume regression, along with a synthetic dataset, to improve body volume estimation accuracy.

Findings

Correctly predicts volume in ~82% of cases within 10% tolerance

Outperforms state-of-the-art solutions like BodyNet

Uses a large-scale synthetic dataset SURREALvols

Abstract

Human body volume estimation from a single RGB image is a challenging problem despite minimal attention from the research community. However VolNet, an architecture leveraging 2D and 3D pose estimation, body part segmentation and volume regression extracted from a single 2D RGB image combined with the subject's body height can be used to estimate the total body volume. VolNet is designed to predict the 2D and 3D pose as well as the body part segmentation in intermediate tasks. We generated a synthetic, large-scale dataset of photo-realistic images of human bodies with a wide range of body shapes and realistic poses called SURREALvols. By using Volnet and combining multiple stacked hourglass networks together with ResNeXt, our model correctly predicted the volume in ~82% of cases with a 10% tolerance threshold. This is a considerable improvement compared to state-of-the-art solutions such as BodyNet with only a ~38% success rate.