SUPER: Selfie Undistortion and Head Pose Editing with Identity Preservation

TL;DR

SUPER is a novel method that corrects distortions and adjusts head pose in close-up selfies by combining 3D GAN inversion, depth estimation, and generative blending, enabling realistic and attractive self-portraits.

Contribution

It introduces a new approach that integrates 3D GAN inversion with depth-based warping and occlusion-aware blending for selfie correction and pose editing.

Findings

Outperforms previous methods on face undistortion benchmarks.

Achieves superior qualitative and quantitative results.

Enables photorealistic selfie editing with preserved identity.

Abstract

Self-portraits captured from a short distance might look unnatural or even unattractive due to heavy distortions making facial features malformed, and ill-placed head poses. In this paper, we propose SUPER, a novel method of eliminating distortions and adjusting head pose in a close-up face crop. We perform 3D GAN inversion for a facial image by optimizing camera parameters and face latent code, which gives a generated image. Besides, we estimate depth from the obtained latent code, create a depth-induced 3D mesh, and render it with updated camera parameters to obtain a warped portrait. Finally, we apply the visibility-based blending so that visible regions are reprojected, and occluded parts are restored with a generative model. Experiments on face undistortion benchmarks and on our self-collected Head Rotation dataset (HeRo), show that SUPER outperforms previous approaches both qualitatively and quantitatively, opening new possibilities for photorealistic selfie editing.