PARTE: Part-Guided Texturing for 3D Human Reconstruction from a Single Image

TL;DR

PARTE introduces a novel framework that leverages 3D human part segmentation to improve the alignment and quality of textures in single-image 3D human reconstructions, addressing misalignment issues present in prior methods.

Contribution

The paper presents a new approach combining part segmentation and part-guided texturing to enhance 3D human reconstruction from a single image, which was not explicitly exploited in previous methods.

Findings

Achieves state-of-the-art quality in 3D human reconstruction

Effectively maintains distinct textures for different human parts

Improves texture alignment and coherence in reconstructed models

Abstract

The misaligned human texture across different human parts is one of the main limitations of existing 3D human reconstruction methods. Each human part, such as a jacket or pants, should maintain a distinct texture without blending into others. The structural coherence of human parts serves as a crucial cue to infer human textures in the invisible regions of a single image. However, most existing 3D human reconstruction methods do not explicitly exploit such part segmentation priors, leading to misaligned textures in their reconstructions. In this regard, we present PARTE, which utilizes 3D human part information as a key guide to reconstruct 3D human textures. Our framework comprises two core components. First, to infer 3D human part information from a single image, we propose a 3D part segmentation module (PartSegmenter) that initially reconstructs a textureless human surface and predicts human part labels based on the textureless surface. Second, to incorporate part information into texture reconstruction, we introduce a part-guided texturing module (PartTexturer), which acquires prior knowledge from a pre-trained image generation network on texture alignment of human parts. Extensive experiments demonstrate that our framework achieves state-of-the-art quality in 3D human reconstruction. The project page is available at https://hygenie1228.github.io/PARTE/.