Unsupervised Portrait Shadow Removal via Generative Priors

TL;DR

This paper introduces the first unsupervised portrait shadow removal method that leverages pretrained StyleGAN2 priors, enabling effective shadow removal without training data, and extends to tattoo and watermark removal.

Contribution

It proposes a novel unsupervised shadow removal approach using generative priors from StyleGAN2, avoiding the need for large-scale training datasets.

Findings

Achieves comparable performance with supervised methods

Effective layer decomposition algorithm for shadow removal

Extends to tattoo and watermark removal tasks

Abstract

Portrait images often suffer from undesirable shadows cast by casual objects or even the face itself. While existing methods for portrait shadow removal require training on a large-scale synthetic dataset, we propose the first unsupervised method for portrait shadow removal without any training data. Our key idea is to leverage the generative facial priors embedded in the off-the-shelf pretrained StyleGAN2. To achieve this, we formulate the shadow removal task as a layer decomposition problem: a shadowed portrait image is constructed by the blending of a shadow image and a shadow-free image. We propose an effective progressive optimization algorithm to learn the decomposition process. Our approach can also be extended to portrait tattoo removal and watermark removal. Qualitative and quantitative experiments on a real-world portrait shadow dataset demonstrate that our approach achieves comparable performance with supervised shadow removal methods. Our source code is available at https://github.com/YingqingHe/Shadow-Removal-via-Generative-Priors.