SARS: A Novel Face and Body Shape and Appearance Aware 3D Reconstruction System extends Morphable Models

TL;DR

This paper introduces SARS, a modular 3D reconstruction system that incorporates face and body shape, appearance, and semantic features from a single image to produce detailed human models, addressing limitations of previous methods.

Contribution

The novel SARS system integrates high-level facial semantic features with shape and appearance data for comprehensive 3D human reconstruction from a single image.

Findings

Effective reconstruction of full human body models.

Inclusion of facial semantic features improves accuracy.

Demonstrates robustness across diverse images.

Abstract

Morphable Models (3DMMs) are a type of morphable model that takes 2D images as inputs and recreates the structure and physical appearance of 3D objects, especially human faces and bodies. 3DMM combines identity and expression blendshapes with a basic face mesh to create a detailed 3D model. The variability in the 3D Morphable models can be controlled by tuning diverse parameters. They are high-level image descriptors, such as shape, texture, illumination, and camera parameters. Previous research in 3D human reconstruction concentrated solely on global face structure or geometry, ignoring face semantic features such as age, gender, and facial landmarks characterizing facial boundaries, curves, dips, and wrinkles. In order to accommodate changes in these high-level facial characteristics, this work introduces a shape and appearance-aware 3D reconstruction system (named SARS by us), a c modular pipeline that extracts body and face information from a single image to properly rebuild the 3D model of the human full body.