PostCam: Camera-Controllable Novel-View Video Generation with Query-Shared Cross-Attention

TL;DR

PostCam is a novel framework that enables post-capture editing of camera views in dynamic videos, improving control precision and visual quality through a query-shared cross-attention mechanism and a two-stage training process.

Contribution

It introduces a query-shared cross-attention module for integrating camera poses and video frames, enabling more accurate and flexible view synthesis in dynamic scenes.

Findings

Outperforms state-of-the-art methods by over 20% in control precision

Achieves higher view consistency and visual fidelity

Effective in both real-world and synthetic datasets

Abstract

We propose PostCam, a framework for novel-view video generation that enables post-capture editing of camera trajectories in dynamic scenes. We find that existing video recapture methods suffer from suboptimal camera motion injection strategies; such suboptimal designs not only limit camera control precision but also result in generated videos that fail to preserve fine visual details from the source video. To achieve more accurate and flexible motion manipulation, PostCam introduces a query-shared cross-attention module. It integrates two distinct forms of control signals: the 6-DoF camera poses and the 2D rendered video frames. By fusing them into a unified representation within a shared feature space, our model can extract underlying motion cues, which enhances both control precision and generation quality. Furthermore, we adopt a two-stage training strategy: the model first learns coarse camera control from pose inputs, and then incorporates visual information to refine motion accuracy and enhance visual fidelity. Experiments on both real-world and synthetic datasets demonstrate that PostCam outperforms state-of-the-art methods by over 20% in camera control precision and view consistency, while achieving the highest video generation quality. Our project webpage is publicly available at: https://cccqaq.github.io/PostCam.github.io/