3D Human Shape Reconstruction from a Polarization Image

TL;DR

This paper introduces a two-stage deep learning method to reconstruct 3D human body shapes from single polarization images, leveraging geometric cues to improve accuracy over traditional imaging methods.

Contribution

It presents a novel approach that uses polarization images and a two-stage network to estimate detailed 3D human shapes, outperforming conventional methods.

Findings

Effective 3D shape reconstruction from polarization images.

Polarization imaging provides rich geometric cues for shape estimation.

Method outperforms traditional color and depth-based approaches.

Abstract

This paper tackles the problem of estimating 3D body shape of clothed humans from single polarized 2D images, i.e. polarization images. Polarization images are known to be able to capture polarized reflected lights that preserve rich geometric cues of an object, which has motivated its recent applications in reconstructing surface normal of the objects of interest. Inspired by the recent advances in human shape estimation from single color images, in this paper, we attempt at estimating human body shapes by leveraging the geometric cues from single polarization images. A dedicated two-stage deep learning approach, SfP, is proposed: given a polarization image, stage one aims at inferring the fined-detailed body surface normal; stage two gears to reconstruct the 3D body shape of clothing details. Empirical evaluations on a synthetic dataset (SURREAL) as well as a real-world dataset (PHSPD) demonstrate the qualitative and quantitative performance of our approach in estimating human poses and shapes. This indicates polarization camera is a promising alternative to the more conventional color or depth imaging for human shape estimation. Further, normal maps inferred from polarization imaging play a significant role in accurately recovering the body shapes of clothed people.