Warp-as-History: Generalizable Camera-Controlled Video Generation from One Training Video

TL;DR

Warp-as-History enables camera-controlled video generation from a single training video without additional training or optimization, by converting camera warps into pseudo-history for improved trajectory following.

Contribution

It introduces a novel interface that leverages camera warps as pseudo-history, enabling zero-shot camera trajectory control in frozen models and enhancing performance with minimal finetuning.

Findings

Zero-shot camera trajectory following achieved without training or optimization.

Lightweight LoRA finetuning improves adherence and visual quality.

Method generalizes well across diverse datasets.

Abstract

Camera-controlled video generation has made substantial progress, enabling generated videos to follow prescribed viewpoint trajectories. However, existing methods usually learn camera-specific conditioning through camera encoders, control branches, or attention and positional-encoding modifications, which often require post-training on large-scale camera-annotated videos. Training-free alternatives avoid such post-training, but often shift the cost to test-time optimization or extra denoising-time guidance. We propose Warp-as-History, a simple interface that turns camera-induced warps into camera-warped pseudo-history with target-frame positional alignment and visible-token selection. Given a target camera trajectory, we construct camera-warped pseudo-history from past observations and feed it through the model's visual-history pathway. Crucially, we align its positional encoding with the target frames being denoised and remove warped-history tokens without valid source observations. Without any training, architectural modification, or test-time optimization, this interface reveals a non-trivial zero-shot capability of a frozen video generation model to follow camera trajectories. Moreover, lightweight offline LoRA finetuning on only one camera-annotated video further improves this capability and generalizes to unseen videos, improving camera adherence, visual quality, and motion dynamics without test-time optimization or target-video adaptation. Extensive experiments on diverse datasets confirm the effectiveness of our method.