SmartEraser: Remove Anything from Images using Masked-Region Guidance

TL;DR

SmartEraser introduces a novel Masked-Region Guidance paradigm for object removal in images, leveraging the masked region as guidance to improve accuracy and context preservation, outperforming existing methods especially in complex scenes.

Contribution

This work proposes the innovative Masked-Region Guidance paradigm and the Syn4Removal dataset, advancing object removal techniques by utilizing masked regions as guidance and providing large-scale training data.

Findings

SmartEraser outperforms existing object removal methods.

The new paradigm improves context preservation and removal accuracy.

Experimental results show superior performance in complex scenes.

Abstract

Object removal has so far been dominated by the mask-and-inpaint paradigm, where the masked region is excluded from the input, leaving models relying on unmasked areas to inpaint the missing region. However, this approach lacks contextual information for the masked area, often resulting in unstable performance. In this work, we introduce SmartEraser, built with a new removing paradigm called Masked-Region Guidance. This paradigm retains the masked region in the input, using it as guidance for the removal process. It offers several distinct advantages: (a) it guides the model to accurately identify the object to be removed, preventing its regeneration in the output; (b) since the user mask often extends beyond the object itself, it aids in preserving the surrounding context in the final result. Leveraging this new paradigm, we present Syn4Removal, a large-scale object removal dataset, where instance segmentation data is used to copy and paste objects onto images as removal targets, with the original images serving as ground truths. Experimental results demonstrate that SmartEraser significantly outperforms existing methods, achieving superior performance in object removal, especially in complex scenes with intricate compositions.