PanFlow: Decoupled Motion Control for Panoramic Video Generation

TL;DR

PanFlow introduces a spherical, decoupled motion control method for panoramic video generation, enabling precise large motion handling and loop consistency, supported by a new dataset and extensive experiments.

Contribution

It presents a novel spherical decoupling approach and spherical noise warping for improved panoramic video synthesis, along with a large-scale annotated dataset.

Findings

Outperforms prior methods in motion fidelity

Enhances visual quality and temporal coherence

Supports applications like motion transfer and editing

Abstract

Panoramic video generation has attracted growing attention due to its applications in virtual reality and immersive media. However, existing methods lack explicit motion control and struggle to generate scenes with large and complex motions. We propose PanFlow, a novel approach that exploits the spherical nature of panoramas to decouple the highly dynamic camera rotation from the input optical flow condition, enabling more precise control over large and dynamic motions. We further introduce a spherical noise warping strategy to promote loop consistency in motion across panorama boundaries. To support effective training, we curate a large-scale, motion-rich panoramic video dataset with frame-level pose and flow annotations. We also showcase the effectiveness of our method in various applications, including motion transfer and video editing. Extensive experiments demonstrate that PanFlow significantly outperforms prior methods in motion fidelity, visual quality, and temporal coherence. Our code, dataset, and models are available at https://github.com/chengzhag/PanFlow.