DNI: Dilutional Noise Initialization for Diffusion Video Editing

TL;DR

This paper introduces Dilutional Noise Initialization (DNI), a novel method that enhances diffusion-based video editing by enabling more flexible, non-rigid edits through a noise dilution technique that softens structural rigidity.

Contribution

The paper proposes a new noise dilution framework that improves the flexibility and precision of diffusion video editing, especially for non-rigid structural modifications.

Findings

DNI significantly improves non-rigid editing capabilities.

Enhanced editing accuracy with closer alignment to target prompts.

Effective structural modifications demonstrated through experiments.

Abstract

Text-based diffusion video editing systems have been successful in performing edits with high fidelity and textual alignment. However, this success is limited to rigid-type editing such as style transfer and object overlay, while preserving the original structure of the input video. This limitation stems from an initial latent noise employed in diffusion video editing systems. The diffusion video editing systems prepare initial latent noise to edit by gradually infusing Gaussian noise onto the input video. However, we observed that the visual structure of the input video still persists within this initial latent noise, thereby restricting non-rigid editing such as motion change necessitating structural modifications. To this end, this paper proposes Dilutional Noise Initialization (DNI) framework which enables editing systems to perform precise and dynamic modification including non-rigid editing. DNI introduces a concept of `noise dilution' which adds further noise to the latent noise in the region to be edited to soften the structural rigidity imposed by input video, resulting in more effective edits closer to the target prompt. Extensive experiments demonstrate the effectiveness of the DNI framework.