Macro-from-Micro Planning for High-Quality and Parallelized Autoregressive Long Video Generation

TL;DR

This paper introduces a hierarchical planning framework for long video generation that improves quality, consistency, and efficiency by combining macro and micro planning stages with parallelized content generation.

Contribution

It proposes a novel Macro-from-Micro Planning framework that enables long, high-quality, and parallelized autoregressive video synthesis, addressing temporal drift and scalability issues.

Findings

Outperforms existing models in video quality and stability

Enables parallelized generation of long videos

Achieves efficient long-term consistency across segments

Abstract

Current autoregressive diffusion models excel at video generation but are generally limited to short temporal durations. Our theoretical analysis indicates that the autoregressive modeling typically suffers from temporal drift caused by error accumulation and hinders parallelization in long video synthesis. To address these limitations, we propose a novel planning-then-populating framework centered on Macro-from-Micro Planning (MMPL) for long video generation. MMPL sketches a global storyline for the entire video through two hierarchical stages: Micro Planning and Macro Planning. Specifically, Micro Planning predicts a sparse set of future keyframes within each short video segment, offering motion and appearance priors to guide high-quality video segment generation. Macro Planning extends the in-segment keyframes planning across the entire video through an autoregressive chain of micro plans, ensuring long-term consistency across video segments. Subsequently, MMPL-based Content Populating generates all intermediate frames in parallel across segments, enabling efficient parallelization of autoregressive generation. The parallelization is further optimized by Adaptive Workload Scheduling for balanced GPU execution and accelerated autoregressive video generation. Extensive experiments confirm that our method outperforms existing long video generation models in quality and stability. Generated videos and comparison results are in our project page.