DCARL: A Divide-and-Conquer Framework for Autoregressive Long-Trajectory Video Generation

TL;DR

DCARL introduces a divide-and-conquer autoregressive framework for long-trajectory video generation, combining structural stability with high-fidelity synthesis, enabling stable, long-duration videos with improved quality and control.

Contribution

The paper presents a novel framework that integrates keyframe generation and interpolation for scalable, high-quality long-trajectory video synthesis, addressing limitations of existing models.

Findings

Achieves superior visual quality with lower FID and FVD scores.

Demonstrates stable generation of videos up to 32 seconds long.

Outperforms state-of-the-art autoregressive and divide-and-conquer baselines.

Abstract

Long-trajectory video generation is a crucial yet challenging task for world modeling primarily due to the limited scalability of existing video diffusion models (VDMs). Autoregressive models, while offering infinite rollout, suffer from visual drift and poor controllability. To address these issues, we propose DCARL, a novel divide-and-conquer, autoregressive framework that effectively combines the structural stability of the divide-and-conquer scheme with the high-fidelity generation of VDMs. Our approach first employs a dedicated Keyframe Generator trained without temporal compression to establish long-range, globally consistent structural anchors. Subsequently, an Interpolation Generator synthesizes the dense frames in an autoregressive manner with overlapping segments, utilizing the keyframes for global context and a single clean preceding frame for local coherence. Trained on a large-scale internet long trajectory video dataset, our method achieves superior performance in both visual quality (lower FID and FVD) and camera adherence (lower ATE and ARE) compared to state-of-the-art autoregressive and divide-and-conquer baselines, demonstrating stable and high-fidelity generation for long trajectory videos up to 32 seconds in length.