LoViC: Efficient Long Video Generation with Context Compression

TL;DR

LoViC introduces a scalable diffusion transformer framework for long video generation by employing a novel context compression method, FlexFormer, enabling efficient, coherent, and versatile long video synthesis.

Contribution

The paper presents FlexFormer, a new autoencoder for variable-length video and text compression, and a segment-wise generation process that scales long video synthesis efficiently.

Findings

Achieves high-quality long video generation with coherent content.

Supports various tasks like prediction, interpolation, and multi-shot generation.

Demonstrates effectiveness across diverse open-domain videos.

Abstract

Despite recent advances in diffusion transformers (DiTs) for text-to-video generation, scaling to long-duration content remains challenging due to the quadratic complexity of self-attention. While prior efforts -- such as sparse attention and temporally autoregressive models -- offer partial relief, they often compromise temporal coherence or scalability. We introduce LoViC, a DiT-based framework trained on million-scale open-domain videos, designed to produce long, coherent videos through a segment-wise generation process. At the core of our approach is FlexFormer, an expressive autoencoder that jointly compresses video and text into unified latent representations. It supports variable-length inputs with linearly adjustable compression rates, enabled by a single query token design based on the Q-Former architecture. Additionally, by encoding temporal context through position-aware mechanisms, our model seamlessly supports prediction, retradiction, interpolation, and multi-shot generation within a unified paradigm. Extensive experiments across diverse tasks validate the effectiveness and versatility of our approach.