A Visual Representation for Editing Face Images

TL;DR

This paper introduces a visual face representation enabling realistic editing of face images, including relighting, makeup transfer, and detail modifications, by decomposing images into geometry, albedo, illumination, and detail maps.

Contribution

It presents a novel visual representation for face editing that preserves identity and realism, outperforming previous methods in relighting and makeup transfer tasks.

Findings

Outperforms existing methods in face relighting and detail editing.

Reliable makeup transfer that enhances attractiveness.

User study confirms improved realism and identity preservation.

Abstract

We propose a new approach for editing face images, which enables numerous exciting applications including face relighting, makeup transfer and face detail editing. Our face edits are based on a visual representation, which includes geometry, face segmentation, albedo, illumination and detail map. To recover our visual representation, we start by estimating geometry using a morphable face model, then decompose the face image to recover the albedo, and then shade the geometry with the albedo and illumination. The residual between our shaded geometry and the input image produces our detail map, which carries high frequency information that is either insufficiently or incorrectly captured by our shading process. By manipulating the detail map, we can edit face images with reality and identity preserved. Our representation allows various applications. First, it allows a user to directly manipulate various illumination. Second, it allows non-parametric makeup transfer with input face's distinctive identity features preserved. Third, it allows non-parametric modifications to the face appearance by transferring details. For face relighting and detail editing, we evaluate via a user study and our method outperforms other methods. For makeup transfer, we evaluate via an online attractiveness evaluation system, and can reliably make people look younger and more attractive. We also show extensive qualitative comparisons to existing methods, and have significant improvements over previous techniques.