LoVoRA: Text-guided and Mask-free Video Object Removal and Addition with Learnable Object-aware Localization

TL;DR

LoVoRA introduces a mask-free, learnable object-aware localization framework for consistent text-guided video object removal and addition, eliminating the need for auxiliary masks or reference images.

Contribution

It presents a novel, end-to-end video editing method leveraging a diffusion mask predictor and a unique dataset pipeline for scalable, high-quality, temporally consistent edits.

Findings

Achieves high-quality, temporally consistent video edits

Eliminates reliance on external masks or reference images

Demonstrates superior performance through extensive experiments

Abstract

Text-guided video editing, particularly for object removal and addition, remains a challenging task due to the need for precise spatial and temporal consistency. Existing methods often rely on auxiliary masks or reference images for editing guidance, which limits their scalability and generalization. To address these issues, we propose LoVoRA, a novel framework for mask-free video object removal and addition using object-aware localization mechanism. Our approach utilizes a unique dataset construction pipeline that integrates image-to-video translation, optical flow-based mask propagation, and video inpainting, enabling temporally consistent edits. The core innovation of LoVoRA is its learnable object-aware localization mechanism, which provides dense spatio-temporal supervision for both object insertion and removal tasks. By leveraging a Diffusion Mask Predictor, LoVoRA achieves end-to-end video editing without requiring external control signals during inference. Extensive experiments and human evaluation demonstrate the effectiveness and high-quality performance of LoVoRA. https://cz-5f.github.io/LoVoRA.github.io