VALA: Learning Latent Anchors for Training-Free and Temporally Consistent

TL;DR

VALA introduces a variational alignment module that adaptively selects key frames and compresses their features into semantic anchors, enhancing temporal consistency and efficiency in training-free video editing with diffusion models.

Contribution

It proposes a novel variational framework with contrastive learning to learn meaningful latent anchors for consistent, training-free video editing.

Findings

Achieves state-of-the-art inversion fidelity and editing quality.

Improves temporal consistency in video editing.

Offers enhanced efficiency over prior methods.

Abstract

Recent advances in training-free video editing have enabled lightweight and precise cross-frame generation by leveraging pre-trained text-to-image diffusion models. However, existing methods often rely on heuristic frame selection to maintain temporal consistency during DDIM inversion, which introduces manual bias and reduces the scalability of end-to-end inference. In this paper, we propose~\textbf{VALA} (\textbf{V}ariational \textbf{A}lignment for \textbf{L}atent \textbf{A}nchors), a variational alignment module that adaptively selects key frames and compresses their latent features into semantic anchors for consistent video editing. To learn meaningful assignments, VALA propose a variational framework with a contrastive learning objective. Therefore, it can transform cross-frame latent representations into compressed latent anchors that preserve both content and temporal coherence. Our method can be fully integrated into training-free text-to-image based video editing models. Extensive experiments on real-world video editing benchmarks show that VALA achieves state-of-the-art performance in inversion fidelity, editing quality, and temporal consistency, while offering improved efficiency over prior methods.