ByeGlassesGAN: Identity Preserving Eyeglasses Removal for Face Images

TL;DR

ByeGlassesGAN is a novel GAN framework that automatically detects and removes eyeglasses from face images, improving visual quality and face recognition accuracy, especially for semi-transparent glasses.

Contribution

The paper introduces ByeGlassesGAN, a new image-to-image GAN with segmentation for eyeglasses removal, enhancing face image quality and recognition performance.

Findings

Effective removal of semi-transparent and glare glasses

Improved face recognition accuracy after glasses removal

Visually appealing results in diverse scenarios

Abstract

In this paper, we propose a novel image-to-image GAN framework for eyeglasses removal, called ByeGlassesGAN, which is used to automatically detect the position of eyeglasses and then remove them from face images. Our ByeGlassesGAN consists of an encoder, a face decoder, and a segmentation decoder. The encoder is responsible for extracting information from the source face image, and the face decoder utilizes this information to generate glasses-removed images. The segmentation decoder is included to predict the segmentation mask of eyeglasses and completed face region. The feature vectors generated by the segmentation decoder are shared with the face decoder, which facilitates better reconstruction results. Our experiments show that ByeGlassesGAN can provide visually appealing results in the eyeglasses-removed face images even for semi-transparent color eyeglasses or glasses with glare. Furthermore, we demonstrate significant improvement in face recognition accuracy for face images with glasses by applying our method as a pre-processing step in our face recognition experiment.