Object-WIPER : Training-Free Object and Associated Effect Removal in Videos

TL;DR

Object-WIPER is a training-free video editing framework that removes objects and their effects while maintaining temporal and semantic consistency, using a pre-trained diffusion model and a new evaluation metric.

Contribution

It introduces a novel training-free method for object and effect removal in videos, leveraging a pre-trained text-to-video diffusion transformer and a new benchmark for evaluation.

Findings

Outperforms existing methods in object removal quality and temporal stability.

Achieves effective removal without retraining or fine-tuning.

Introduces a new benchmark for evaluating object removal in videos.

Abstract

In this paper, we introduce Object-WIPER, a training-free framework for removing dynamic objects and their associated visual effects from videos, and inpainting them with semantically consistent and temporally coherent content. Our approach leverages a pre-trained text-to-video diffusion transformer (DiT). Given an input video, a user-provided object mask, and query tokens describing the target object and its effects, we localize relevant visual tokens via visual-text cross-attention and visual self-attention. This produces an intermediate effect mask that we fuse with the user mask to obtain a final foreground token mask to replace. We first invert the video through the DiT to obtain structured noise, then reinitialize the masked tokens with Gaussian noise while preserving background tokens. During denoising, we copy values for the background tokens saved during inversion to maintain scene fidelity. To address the lack of suitable evaluation, we introduce a new object removal metric that rewards temporal consistency among foreground tokens across consecutive frames, coherence between foreground and background tokens within each frame, and dissimilarity between the input and output foreground tokens. Experiments on DAVIS and a newly curated real-world associated effect benchmark (WIPER-Bench) show that Object-WIPER surpasses both training-based and training-free baselines in terms of the metric, achieving clean removal and temporally stable reconstruction without any retraining. Our new benchmark, source code, and pre-trained models will be publicly available.