Generative Landmarks Guided Eyeglasses Removal 3D Face Reconstruction

TL;DR

This paper introduces a novel deep learning-based method for 3D face reconstruction from a single image that effectively removes eyeglasses, ensuring realistic and topologically accurate facial models even in unconstrained, in-the-wild conditions.

Contribution

The proposed approach robustly identifies and removes eyeglasses during 3D face reconstruction, incorporating face parsing to enhance realism and topological accuracy in challenging scenarios.

Findings

Outperforms existing methods in in-the-wild conditions

Successfully removes eyeglasses for realistic 3D face models

Demonstrates superior regulation ability in experiments

Abstract

Single-view 3D face reconstruction is a fundamental Computer Vision problem of extraordinary difficulty. Current systems often assume the input is unobstructed faces which makes their method not suitable for in-the-wild conditions. We present a method for performing a 3D face that removes eyeglasses from a single image. Existing facial reconstruction methods fail to remove eyeglasses automatically for generating a photo-realistic 3D face "in-the-wild".The innovation of our method lies in a process for identifying the eyeglasses area robustly and remove it intelligently. In this work, we estimate the 2D face structure of the reasonable position of the eyeglasses area, which is used for the construction of 3D texture. An excellent anti-eyeglasses face reconstruction method should ensure the authenticity of the output, including the topological structure between the eyes, nose, and mouth. We achieve this via a deep learning architecture that performs direct regression of a 3DMM representation of the 3D facial geometry from a single 2D image. We also demonstrate how the related face parsing task can be incorporated into the proposed framework and help improve reconstruction quality. We conduct extensive experiments on existing 3D face reconstruction tasks as concrete examples to demonstrate the method's superior regulation ability over existing methods often break down.