FADE: Frequency-Aware Diffusion Model Factorization for Video Editing

TL;DR

FADE is a novel, training-free video editing method that leverages frequency-aware factorization of pre-trained diffusion models to enable efficient, high-quality, and temporally coherent edits without extensive computational costs.

Contribution

The paper introduces a frequency-aware factorization strategy and spectrum-guided modulation for effective video editing using pre-trained diffusion models, without additional training.

Findings

Achieves high-quality, realistic video edits with temporal coherence.

Supports versatile edits while preserving spatial and temporal structures.

Demonstrates effectiveness through extensive experiments on real-world videos.

Abstract

Recent advancements in diffusion frameworks have significantly enhanced video editing, achieving high fidelity and strong alignment with textual prompts. However, conventional approaches using image diffusion models fall short in handling video dynamics, particularly for challenging temporal edits like motion adjustments. While current video diffusion models produce high-quality results, adapting them for efficient editing remains difficult due to the heavy computational demands that prevent the direct application of previous image editing techniques. To overcome these limitations, we introduce FADE, a training-free yet highly effective video editing approach that fully leverages the inherent priors from pre-trained video diffusion models via frequency-aware factorization. Rather than simply using these models, we first analyze the attention patterns within the video model to reveal how video priors are distributed across different components. Building on these insights, we propose a factorization strategy to optimize each component's specialized role. Furthermore, we devise spectrum-guided modulation to refine the sampling trajectory with frequency domain cues, preventing information leakage and supporting efficient, versatile edits while preserving the basic spatial and temporal structure. Extensive experiments on real-world videos demonstrate that our method consistently delivers high-quality, realistic and temporally coherent editing results both qualitatively and quantitatively. Code is available at https://github.com/EternalEvan/FADE .