CamCloneMaster: Enabling Reference-based Camera Control for Video Generation

TL;DR

CamCloneMaster introduces an intuitive reference-based camera control framework for video generation, allowing users to replicate complex camera movements from reference videos without explicit parameters or fine-tuning.

Contribution

It presents a novel unified framework for reference-based camera control in video generation and introduces a large-scale synthetic dataset for camera clone learning.

Findings

Outperforms existing methods in controllability and visual quality

Supports both Image-to-Video and Video-to-Video tasks

Validated by extensive experiments and user studies

Abstract

Camera control is crucial for generating expressive and cinematic videos. Existing methods rely on explicit sequences of camera parameters as control conditions, which can be cumbersome for users to construct, particularly for intricate camera movements. To provide a more intuitive camera control method, we propose CamCloneMaster, a framework that enables users to replicate camera movements from reference videos without requiring camera parameters or test-time fine-tuning. CamCloneMaster seamlessly supports reference-based camera control for both Image-to-Video and Video-to-Video tasks within a unified framework. Furthermore, we present the Camera Clone Dataset, a large-scale synthetic dataset designed for camera clone learning, encompassing diverse scenes, subjects, and camera movements. Extensive experiments and user studies demonstrate that CamCloneMaster outperforms existing methods in terms of both camera controllability and visual quality.