Synthesizing Human Faces using Latent Space Factorization and Local Weights (Extended Version)

TL;DR

This paper introduces a 3D face generative model that uses latent space factorization and local weights to enable detailed facial part manipulation and enhance the model's expressiveness.

Contribution

It presents a novel approach combining latent space factorization with local weights via non-negative matrix factorization for improved facial manipulation.

Findings

Facial part manipulation is effectively achieved.

Model's expressiveness is significantly improved.

Semantic meaningfulness of parts is validated.

Abstract

We propose a 3D face generative model with local weights to increase the model's variations and expressiveness. The proposed model allows partial manipulation of the face while still learning the whole face mesh. For this purpose, we address an effective way to extract local facial features from the entire data and explore a way to manipulate them during a holistic generation. First, we factorize the latent space of the whole face to the subspace indicating different parts of the face. In addition, local weights generated by non-negative matrix factorization are applied to the factorized latent space so that the decomposed part space is semantically meaningful. We experiment with our model and observe that effective facial part manipulation is possible and that the model's expressiveness is improved.