Mask-Guided Image Person Removal with Data Synthesis

TL;DR

This paper introduces a novel data synthesis and learning framework for image person removal, addressing dataset scarcity and complex human postures, achieving effective results on real and synthetic images.

Contribution

It proposes new dataset generation methods and a guided learning framework for improved person removal in images, with a coarse-to-fine training strategy.

Findings

Effective removal of persons demonstrated on real images

High-quality results with detailed texture preservation

Good generalization to synthetic and real images

Abstract

As a special case of common object removal, image person removal is playing an increasingly important role in social media and criminal investigation domains. Due to the integrity of person area and the complexity of human posture, person removal has its own dilemmas. In this paper, we propose a novel idea to tackle these problems from the perspective of data synthesis. Concerning the lack of dedicated dataset for image person removal, two dataset production methods are proposed to automatically generate images, masks and ground truths respectively. Then, a learning framework similar to local image degradation is proposed so that the masks can be used to guide the feature extraction process and more texture information can be gathered for final prediction. A coarse-to-fine training strategy is further applied to refine the details. The data synthesis and learning framework combine well with each other. Experimental results verify the effectiveness of our method quantitatively and qualitatively, and the trained network proves to have good generalization ability either on real or synthetic images.