You Only Erase Once: Erasing Anything without Bringing Unexpected Content

TL;DR

The paper introduces YOEO, a diffusion-based object erasure method that produces high-quality results without unwanted artifacts, using unpaired data and a diffusion distillation strategy.

Contribution

YOEO is a novel object erasure approach that overcomes limitations of previous methods by training on unpaired data and employing diffusion distillation for efficiency.

Findings

Outperforms state-of-the-art object erasure methods.

Produces artifact-free, context-coherent erasure results.

Uses unpaired real-world images for training.

Abstract

We present YOEO, an approach for object erasure. Unlike recent diffusion-based methods which struggle to erase target objects without generating unexpected content within the masked regions due to lack of sufficient paired training data and explicit constraint on content generation, our method allows to produce high-quality object erasure results free of unwanted objects or artifacts while faithfully preserving the overall context coherence to the surrounding content. We achieve this goal by training an object erasure diffusion model on unpaired data containing only large-scale real-world images, under the supervision of a sundries detector and a context coherence loss that are built upon an entity segmentation model. To enable more efficient training and inference, a diffusion distillation strategy is employed to train for a few-step erasure diffusion model. Extensive experiments show that our method outperforms the state-of-the-art object erasure methods. Code will be available at https://zyxunh.github.io/YOEO-ProjectPage/.