Human Pose and Shape Estimation from Single Polarization Images

TL;DR

This paper introduces a novel method for estimating human pose and shape from single polarization images using a two-stage pipeline, validated on a new large dataset, showing polarization cameras as a promising alternative to RGB cameras.

Contribution

The paper presents a new polarization-based approach with a dedicated dataset for human pose and shape estimation, advancing beyond traditional RGB methods.

Findings

Effective human pose and shape estimation from polarization images.

A new dataset with over 500K frames for training and evaluation.

Polarization cameras outperform RGB in certain geometric cue extraction.

Abstract

This paper focuses on a new problem of estimating human pose and shape from single polarization images. Polarization camera is known to be able to capture the polarization of reflected lights that preserves rich geometric cues of an object surface. Inspired by the recent applications in surface normal reconstruction from polarization images, in this paper, we attempt to estimate human pose and shape from single polarization images by leveraging the polarization-induced geometric cues. A dedicated two-stage pipeline is proposed: given a single polarization image, stage one (Polar2Normal) focuses on the fine detailed human body surface normal estimation; stage two (Polar2Shape) then reconstructs clothed human shape from the polarization image and the estimated surface normal. To empirically validate our approach, a dedicated dataset (PHSPD) is constructed, consisting of over 500K frames with accurate pose and parametric shape annotations. Empirical evaluations on this real-world dataset as well as a synthetic dataset, SURREAL, demonstrate the effectiveness of our approach. It suggests polarization camera as a promising alternative to the more conventional RGB camera for human pose and shape estimation.