DiViD: Disentangled Video Diffusion for Static-Dynamic Factorization

TL;DR

DiViD is a novel end-to-end video diffusion framework that explicitly disentangles static appearance from dynamic motion, improving accuracy and reducing leakage in video representations.

Contribution

It introduces a new diffusion-based approach with explicit static-dynamic factorization, including a sequence encoder, specialized decoder, and regularizer, outperforming prior methods.

Findings

Achieves highest swap-based joint accuracy

Preserves static fidelity during dynamic transfer

Reduces cross-leakage compared to state-of-the-art methods

Abstract

Unsupervised disentanglement of static appearance and dynamic motion in video remains a fundamental challenge, often hindered by information leakage and blurry reconstructions in existing VAE- and GAN-based approaches. We introduce DiViD, the first end-to-end video diffusion framework for explicit static-dynamic factorization. DiViD's sequence encoder extracts a global static token from the first frame and per-frame dynamic tokens, explicitly removing static content from the motion code. Its conditional DDPM decoder incorporates three key inductive biases: a shared-noise schedule for temporal consistency, a time-varying KL-based bottleneck that tightens at early timesteps (compressing static information) and relaxes later (enriching dynamics), and cross-attention that routes the global static token to all frames while keeping dynamic tokens frame-specific. An orthogonality regularizer further prevents residual static-dynamic leakage. We evaluate DiViD on real-world benchmarks using swap-based accuracy and cross-leakage metrics. DiViD outperforms state-of-the-art sequential disentanglement methods: it achieves the highest swap-based joint accuracy, preserves static fidelity while improving dynamic transfer, and reduces average cross-leakage.