SMPLitex: A Generative Model and Dataset for 3D Human Texture Estimation from Single Image

TL;DR

SMPLitex is a novel generative model and dataset that enables detailed 3D human texture estimation and manipulation from a single image, outperforming existing methods in quality and versatility.

Contribution

It introduces a new generative model for 3D human appearance and a high-quality dataset, advancing single-image 3D human texture estimation and editing capabilities.

Findings

Outperforms existing methods in texture estimation accuracy

Enables diverse tasks like editing and synthesis

Validated on three public datasets

Abstract

We propose SMPLitex, a method for estimating and manipulating the complete 3D appearance of humans captured from a single image. SMPLitex builds upon the recently proposed generative models for 2D images, and extends their use to the 3D domain through pixel-to-surface correspondences computed on the input image. To this end, we first train a generative model for complete 3D human appearance, and then fit it into the input image by conditioning the generative model to the visible parts of the subject. Furthermore, we propose a new dataset of high-quality human textures built by sampling SMPLitex conditioned on subject descriptions and images. We quantitatively and qualitatively evaluate our method in 3 publicly available datasets, demonstrating that SMPLitex significantly outperforms existing methods for human texture estimation while allowing for a wider variety of tasks such as editing, synthesis, and manipulation