Reference-guided Face Component Editing

TL;DR

The paper introduces r-FACE, a flexible face editing framework that uses reference images to control facial component shapes, overcoming limitations of previous attribute-based or manual mask-based methods.

Contribution

It proposes a novel reference-guided face component editing framework utilizing an inpainting backbone and attention modules for precise shape control.

Findings

Effective control of facial component shapes demonstrated

Outperforms existing methods in flexibility and quality

Validated through extensive experiments

Abstract

Face portrait editing has achieved great progress in recent years. However, previous methods either 1) operate on pre-defined face attributes, lacking the flexibility of controlling shapes of high-level semantic facial components (e.g., eyes, nose, mouth), or 2) take manually edited mask or sketch as an intermediate representation for observable changes, but such additional input usually requires extra efforts to obtain. To break the limitations (e.g. shape, mask or sketch) of the existing methods, we propose a novel framework termed r-FACE (Reference-guided FAce Component Editing) for diverse and controllable face component editing with geometric changes. Specifically, r-FACE takes an image inpainting model as the backbone, utilizing reference images as conditions for controlling the shape of face components. In order to encourage the framework to concentrate on the target face components, an example-guided attention module is designed to fuse attention features and the target face component features extracted from the reference image. Through extensive experimental validation and comparisons, we verify the effectiveness of the proposed framework.