Identity-preserving Face Recovery from Portraits

TL;DR

This paper introduces IFRP, a novel method for recovering photorealistic faces from stylized portraits while preserving identity, using a dual-network approach with spatial transformers and feature-based identity constraints.

Contribution

The paper proposes IFRP, combining style removal and discriminative networks with spatial transformers and feature similarity measures to improve identity-preserving face recovery from stylized images.

Findings

Achieves state-of-the-art results on synthesized datasets.

Successfully recovers faces from unseen stylized portraits, paintings, and sketches.

Outperforms existing methods in identity preservation and image quality.

Abstract

Recovering the latent photorealistic faces from their artistic portraits aids human perception and facial analysis. However, a recovery process that can preserve identity is challenging because the fine details of real faces can be distorted or lost in stylized images. In this paper, we present a new Identity-preserving Face Recovery from Portraits (IFRP) to recover latent photorealistic faces from unaligned stylized portraits. Our IFRP method consists of two components: Style Removal Network (SRN) and Discriminative Network (DN). The SRN is designed to transfer feature maps of stylized images to the feature maps of the corresponding photorealistic faces. By embedding spatial transformer networks into the SRN, our method can compensate for misalignments of stylized faces automatically and output aligned realistic face images. The role of the DN is to enforce recovered faces to be similar to authentic faces. To ensure the identity preservation, we promote the recovered and ground-truth faces to share similar visual features via a distance measure which compares features of recovered and ground-truth faces extracted from a pre-trained VGG network. We evaluate our method on a large-scale synthesized dataset of real and stylized face pairs and attain state of the art results. In addition, our method can recover photorealistic faces from previously unseen stylized portraits, original paintings and human-drawn sketches.