Sphere Face Model:A 3D Morphable Model with Hypersphere Manifold Latent Space

TL;DR

The paper introduces the Sphere Face Model, a 3D morphable model with a hypersphere latent space that improves face shape fidelity and identity consistency in monocular face reconstruction.

Contribution

It proposes a novel 3DMM with a hyperspherical latent space and a two-stage training approach to enhance shape fidelity and identity preservation.

Findings

High representation ability demonstrated

Effective shape clustering achieved

Produces consistent and high-fidelity face shapes

Abstract

3D Morphable Models (3DMMs) are generative models for face shape and appearance. However, the shape parameters of traditional 3DMMs satisfy the multivariate Gaussian distribution while the identity embeddings satisfy the hypersphere distribution, and this conflict makes it challenging for face reconstruction models to preserve the faithfulness and the shape consistency simultaneously. To address this issue, we propose the Sphere Face Model(SFM), a novel 3DMM for monocular face reconstruction, which can preserve both shape fidelity and identity consistency. The core of our SFM is the basis matrix which can be used to reconstruct 3D face shapes, and the basic matrix is learned by adopting a two-stage training approach where 3D and 2D training data are used in the first and second stages, respectively. To resolve the distribution mismatch, we design a novel loss to make the shape parameters have a hyperspherical latent space. Extensive experiments show that SFM has high representation ability and shape parameter space's clustering performance. Moreover, it produces fidelity face shapes, and the shapes are consistent in challenging conditions in monocular face reconstruction.