Shaping a Stabilized Video by Mitigating Unintended Changes for Concept-Augmented Video Editing

TL;DR

This paper introduces a novel video editing method that improves stability and fidelity by using concept-augmented textual inversion and dual prior supervision, enabling more nuanced and stable text-driven video edits.

Contribution

It proposes a new framework combining concept-augmented textual inversion with dual prior supervision to enhance video editing stability and attribute control.

Findings

Produces more stable and lifelike videos

Outperforms state-of-the-art methods in stability and fidelity

Enables flexible, stylized video editing

Abstract

Text-driven video editing utilizing generative diffusion models has garnered significant attention due to their potential applications. However, existing approaches are constrained by the limited word embeddings provided in pre-training, which hinders nuanced editing targeting open concepts with specific attributes. Directly altering the keywords in target prompts often results in unintended disruptions to the attention mechanisms. To achieve more flexible editing easily, this work proposes an improved concept-augmented video editing approach that generates diverse and stable target videos flexibly by devising abstract conceptual pairs. Specifically, the framework involves concept-augmented textual inversion and a dual prior supervision mechanism. The former enables plug-and-play guidance of stable diffusion for video editing, effectively capturing target attributes for more stylized results. The dual prior supervision mechanism significantly enhances video stability and fidelity. Comprehensive evaluations demonstrate that our approach generates more stable and lifelike videos, outperforming state-of-the-art methods.