Reconstructing Recognizable 3D Face Shapes based on 3D Morphable Models

TL;DR

This paper introduces a novel regularization technique and training strategy to improve the visual discriminability of reconstructed 3D face shapes from shape parameters, outperforming existing methods.

Contribution

It proposes a shape identity-aware regularization loss and a training approach to enhance the visual and identity discrimination of 3D face reconstructions.

Findings

Improved reconstruction error over state-of-the-art methods

Enhanced visual distinguishability of 3D face shapes

Higher face recognition accuracy using shape parameters

Abstract

Many recent works have reconstructed distinctive 3D face shapes by aggregating shape parameters of the same identity and separating those of different people based on parametric models (e.g., 3D morphable models (3DMMs)). However, despite the high accuracy in the face recognition task using these shape parameters, the visual discrimination of face shapes reconstructed from those parameters is unsatisfactory. The following research question has not been answered in previous works: Do discriminative shape parameters guarantee visual discrimination in represented 3D face shapes? This paper analyzes the relationship between shape parameters and reconstructed shape geometry and proposes a novel shape identity-aware regularization(SIR) loss for shape parameters, aiming at increasing discriminability in both the shape parameter and shape geometry domains. Moreover, to cope with the lack of training data containing both landmark and identity annotations, we propose a network structure and an associated training strategy to leverage mixed data containing either identity or landmark labels. We compare our method with existing methods in terms of the reconstruction error, visual distinguishability, and face recognition accuracy of the shape parameters. Experimental results show that our method outperforms the state-of-the-art methods.