Ouroboros-Diffusion: Exploring Consistent Content Generation in Tuning-free Long Video Diffusion

TL;DR

Ouroboros-Diffusion is a novel video denoising framework that enhances structural and subject consistency for long video generation, addressing limitations of FIFO-Diffusion in maintaining long-range temporal coherence.

Contribution

It introduces a new latent sampling technique, Subject-Aware Cross-Frame Attention, and self-recurrent guidance to improve long video consistency and subject coherence.

Findings

Outperforms existing methods on VBench in subject consistency

Achieves smoother motion and better temporal coherence

Enhances structural and content consistency in long videos

Abstract

The first-in-first-out (FIFO) video diffusion, built on a pre-trained text-to-video model, has recently emerged as an effective approach for tuning-free long video generation. This technique maintains a queue of video frames with progressively increasing noise, continuously producing clean frames at the queue's head while Gaussian noise is enqueued at the tail. However, FIFO-Diffusion often struggles to keep long-range temporal consistency in the generated videos due to the lack of correspondence modeling across frames. In this paper, we propose Ouroboros-Diffusion, a novel video denoising framework designed to enhance structural and content (subject) consistency, enabling the generation of consistent videos of arbitrary length. Specifically, we introduce a new latent sampling technique at the queue tail to improve structural consistency, ensuring perceptually smooth transitions among frames. To enhance subject consistency, we devise a Subject-Aware Cross-Frame Attention (SACFA) mechanism, which aligns subjects across frames within short segments to achieve better visual coherence. Furthermore, we introduce self-recurrent guidance. This technique leverages information from all previous cleaner frames at the front of the queue to guide the denoising of noisier frames at the end, fostering rich and contextual global information interaction. Extensive experiments of long video generation on the VBench benchmark demonstrate the superiority of our Ouroboros-Diffusion, particularly in terms of subject consistency, motion smoothness, and temporal consistency.