Landmark Enforcement and Style Manipulation for Generative Morphing

TL;DR

This paper introduces a novel StyleGAN-based morph generation method with landmark enforcement and latent domain averaging to improve identity preservation and high-frequency details in face morphs, enhancing facial recognition attack effectiveness.

Contribution

It proposes a landmark enforcement technique and latent space exploration to improve identity retention and detail in GAN-based face morphing, addressing limitations of existing methods.

Findings

Enhanced identity preservation in generated morphs.

Improved high-frequency detail reconstruction.

Effective manipulation of morphing style via latent space analysis.

Abstract

Morph images threaten Facial Recognition Systems (FRS) by presenting as multiple individuals, allowing an adversary to swap identities with another subject. Morph generation using generative adversarial networks (GANs) results in high-quality morphs unaffected by the spatial artifacts caused by landmark-based methods, but there is an apparent loss in identity with standard GAN-based morphing methods. In this paper, we propose a novel StyleGAN morph generation technique by introducing a landmark enforcement method to resolve this issue. Considering this method, we aim to enforce the landmarks of the morph image to represent the spatial average of the landmarks of the bona fide faces and subsequently the morph images to inherit the geometric identity of both bona fide faces. Exploration of the latent space of our model is conducted using Principal Component Analysis (PCA) to accentuate the effect of both the bona fide faces on the morphed latent representation and address the identity loss issue with latent domain averaging. Additionally, to improve high frequency reconstruction in the morphs, we study the train-ability of the noise input for the StyleGAN2 model.