FReeNet: Multi-Identity Face Reenactment

TL;DR

FReeNet is a novel multi-identity face reenactment framework that transfers facial expressions between arbitrary identities using a unified model, achieving photorealistic results with enhanced facial detail.

Contribution

The paper introduces FReeNet, combining a landmark converter and geometry-aware generator, with a new triplet perceptual loss for improved face reenactment quality.

Findings

Produces photorealistic, expression-alike faces

Enables flexible expression transfer between identities

Outperforms existing methods in quality and realism

Abstract

This paper presents a novel multi-identity face reenactment framework, named FReeNet, to transfer facial expressions from an arbitrary source face to a target face with a shared model. The proposed FReeNet consists of two parts: Unified Landmark Converter (ULC) and Geometry-aware Generator (GAG). The ULC adopts an encode-decoder architecture to efficiently convert expression in a latent landmark space, which significantly narrows the gap of the face contour between source and target identities. The GAG leverages the converted landmark to reenact the photorealistic image with a reference image of the target person. Moreover, a new triplet perceptual loss is proposed to force the GAG module to learn appearance and geometry information simultaneously, which also enriches facial details of the reenacted images. Further experiments demonstrate the superiority of our approach for generating photorealistic and expression-alike faces, as well as the flexibility for transferring facial expressions between identities.