Pixel Sampling for Style Preserving Face Pose Editing

TL;DR

This paper introduces a novel pixel sampling method for face pose editing that preserves identity and image style, enabling flexible 3D pose manipulation with improved results over existing methods.

Contribution

A two-stage face pose editing approach using pixel attention sampling and 3D landmarks, enhancing style preservation and pose control beyond yaw-only adjustments.

Findings

Faithful style preservation in face editing

Effective 3D pose manipulation including yaw, pitch, and roll

Superior qualitative and quantitative results

Abstract

The existing auto-encoder based face pose editing methods primarily focus on modeling the identity preserving ability during pose synthesis, but are less able to preserve the image style properly, which refers to the color, brightness, saturation, etc. In this paper, we take advantage of the well-known frontal/profile optical illusion and present a novel two-stage approach to solve the aforementioned dilemma, where the task of face pose manipulation is cast into face inpainting. By selectively sampling pixels from the input face and slightly adjust their relative locations with the proposed ``Pixel Attention Sampling" module, the face editing result faithfully keeps the identity information as well as the image style unchanged. By leveraging high-dimensional embedding at the inpainting stage, finer details are generated. Further, with the 3D facial landmarks as guidance, our method is able to manipulate face pose in three degrees of freedom, i.e., yaw, pitch, and roll, resulting in more flexible face pose editing than merely controlling the yaw angle as usually achieved by the current state-of-the-art. Both the qualitative and quantitative evaluations validate the superiority of the proposed approach.