Task-Decoupled Image Inpainting Framework for Class-specific Object Remover

TL;DR

This paper introduces a task-decoupled framework for class-specific object removal in images, training separate models for object restoration and removal to improve removal quality over traditional inpainting methods.

Contribution

It proposes a novel task-decoupled inpainting framework with separate models for restoration and removal, and a data curation method for training class-specific object removers.

Findings

The class-specific object remover outperforms general inpainting-based removers.

The framework effectively isolates object removal and restoration tasks.

Experimental results demonstrate improved removal accuracy on multiple datasets.

Abstract

Object removal refers to the process of erasing designated objects from an image while preserving the overall appearance. Existing works on object removal erase removal targets using image inpainting networks. However, image inpainting networks often generate unsatisfactory removal results. In this work, we find that the current training approach which encourages a single image inpainting model to handle both object removal and restoration tasks is one of the reasons behind such unsatisfactory result. Based on this finding, we propose a task-decoupled image inpainting framework which generates two separate inpainting models: an object restorer for object restoration tasks and an object remover for object removal tasks. We train the object restorer with the masks that partially cover the removal targets. Then, the proposed framework makes an object restorer to generate a guidance for training the object remover. Using the proposed framework, we obtain a class-specific object remover which focuses on removing objects of a target class, aiming to better erase target class objects than general object removers. We also introduce a data curation method that encompasses the image selection and mask generation approaches used to produce training data for the proposed class-specific object remover. Using the proposed curation method, we can simulate the scenarios where an object remover is trained on the data with object removal ground truth images. Experiments on multiple datasets show that the proposed class-specific object remover can better remove target class objects than object removers based on image inpainting networks.