HoloCine: Holistic Generation of Cinematic Multi-Shot Long Video Narratives

TL;DR

HoloCine introduces a holistic model for generating long, coherent cinematic videos with global consistency, directorial control, and emergent cinematic understanding, advancing automated filmmaking.

Contribution

It presents a novel architecture with Window Cross-Attention and Sparse Inter-Shot Self-Attention for coherent, long-form video narrative generation, surpassing previous clip-based models.

Findings

Achieves state-of-the-art narrative coherence in long videos

Develops persistent memory for characters and scenes

Demonstrates emergent cinematic techniques understanding

Abstract

State-of-the-art text-to-video models excel at generating isolated clips but fall short of creating the coherent, multi-shot narratives, which are the essence of storytelling. We bridge this "narrative gap" with HoloCine, a model that generates entire scenes holistically to ensure global consistency from the first shot to the last. Our architecture achieves precise directorial control through a Window Cross-Attention mechanism that localizes text prompts to specific shots, while a Sparse Inter-Shot Self-Attention pattern (dense within shots but sparse between them) ensures the efficiency required for minute-scale generation. Beyond setting a new state-of-the-art in narrative coherence, HoloCine develops remarkable emergent abilities: a persistent memory for characters and scenes, and an intuitive grasp of cinematic techniques. Our work marks a pivotal shift from clip synthesis towards automated filmmaking, making end-to-end cinematic creation a tangible future. Our code is available at: https://holo-cine.github.io/.