STAGE: Storyboard-Anchored Generation for Cinematic Multi-shot Narrative

TL;DR

STAGE introduces a storyboard-anchored workflow for multi-shot cinematic video generation, improving narrative coherence and cinematic quality through novel structural prediction and memory mechanisms.

Contribution

The paper proposes a new storyboard-based framework with innovative memory and encoding strategies, along with a large annotated dataset for cinematic multi-shot video synthesis.

Findings

Outperforms existing methods in narrative control and coherence

Achieves high-quality cinematic multi-shot video generation

Provides a new dataset with detailed annotations for story and cinematic attributes

Abstract

While recent advancements in generative models have achieved remarkable visual fidelity in video synthesis, creating coherent multi-shot narratives remains a significant challenge. To address this, keyframe-based approaches have emerged as a promising alternative to computationally intensive end-to-end methods, offering the advantages of fine-grained control and greater efficiency. However, these methods often fail to maintain cross-shot consistency and capture cinematic language. In this paper, we introduce STAGE, a SToryboard-Anchored GEneration workflow to reformulate the keyframe-based multi-shot video generation task. Instead of using sparse keyframes, we propose STEP2 to predict a structural storyboard composed of start-end frame pairs for each shot. We introduce the multi-shot memory pack to ensure long-range entity consistency, the dual-encoding strategy for intra-shot coherence, and the two-stage training scheme to learn cinematic inter-shot transition. We also contribute the large-scale ConStoryBoard dataset, including high-quality movie clips with fine-grained annotations for story progression, cinematic attributes, and human preferences. Extensive experiments demonstrate that STAGE achieves superior performance in structured narrative control and cross-shot coherence. Our code will be available at this url.