Pixel2ISDF: Implicit Signed Distance Fields based Human Body Model from Multi-view and Multi-pose Images

TL;DR

This paper introduces Pixel2ISDF, a method for reconstructing clothed human bodies in a canonical space using multi-view and multi-pose images, leveraging implicit SDFs and SMPLX priors for improved geometry accuracy.

Contribution

It proposes a novel approach combining implicit SDFs with SMPLX-based latent codes to enhance human body reconstruction from multiple images.

Findings

Achieved 3rd place in WCPA MVP-Human Body Challenge.

Effectively generalizes to unseen images using normal maps.

Utilizes multi-view features for accurate canonical space reconstruction.

Abstract

In this report, we focus on reconstructing clothed humans in the canonical space given multiple views and poses of a human as the input. To achieve this, we utilize the geometric prior of the SMPLX model in the canonical space to learn the implicit representation for geometry reconstruction. Based on the observation that the topology between the posed mesh and the mesh in the canonical space are consistent, we propose to learn latent codes on the posed mesh by leveraging multiple input images and then assign the latent codes to the mesh in the canonical space. Specifically, we first leverage normal and geometry networks to extract the feature vector for each vertex on the SMPLX mesh. Normal maps are adopted for better generalization to unseen images compared to 2D images. Then, features for each vertex on the posed mesh from multiple images are integrated by MLPs. The integrated features acting as the latent code are anchored to the SMPLX mesh in the canonical space. Finally, latent code for each 3D point is extracted and utilized to calculate the SDF. Our work for reconstructing the human shape on canonical pose achieves 3rd performance on WCPA MVP-Human Body Challenge.