PRIMEdit: Probability Redistribution for Instance-aware Multi-object Video Editing with Benchmark Dataset

TL;DR

PRIMEdit is a novel zero-shot framework for precise, instance-aware multi-object video editing that introduces probability redistribution and a new dataset, significantly improving editing fidelity and leakage prevention.

Contribution

The paper proposes PRIMEdit, a zero-shot multi-object video editing method with novel modules and a new dataset, advancing localized editing accuracy and evaluation.

Findings

Outperforms state-of-the-art in editing faithfulness

Reduces editing leakage effectively

Sets new benchmark with the MIVE dataset

Abstract

Recent AI-based video editing has enabled users to edit videos through simple text prompts, significantly simplifying the editing process. However, recent zero-shot video editing techniques primarily focus on global or single-object edits, which can lead to unintended changes in other parts of the video. When multiple objects require localized edits, existing methods face challenges, such as unfaithful editing, editing leakage, and lack of suitable evaluation datasets and metrics. To overcome these limitations, we propose $\textbf{P}$robability $\textbf{R}$edistribution for $\textbf{I}$nstance-aware $\textbf{M}$ulti-object Video $\textbf{Edit}$ing ($\textbf{PRIMEdit}$). PRIMEdit is a zero-shot framework that introduces two key modules: (i) Instance-centric Probability Redistribution (IPR) to ensure precise localization and faithful editing and (ii) Disentangled Multi-instance Sampling (DMS) to prevent editing leakage. Additionally, we present our new MIVE Dataset for video editing featuring diverse video scenarios, and introduce the Cross-Instance Accuracy (CIA) Score to evaluate editing leakage in multi-instance video editing tasks. Our extensive qualitative, quantitative, and user study evaluations demonstrate that PRIMEdit significantly outperforms recent state-of-the-art methods in terms of editing faithfulness, accuracy, and leakage prevention, setting a new benchmark for multi-instance video editing.