CRePE: Curved Ray Expectation Positional Encoding for Unified-Camera-Controlled Video Generation

TL;DR

CRePE introduces a novel camera-aware positional encoding for video generation that effectively handles wide-angle and fisheye lenses, enhancing stability and geometry-awareness in camera control.

Contribution

The paper proposes CRePE, a unified camera model-compatible positional encoding that incorporates depth-aware geometry, improving camera control stability and supporting external geometry control.

Findings

CRePE improves stability in camera control across diverse camera models.

It enhances geometry-aware and perceptual-quality metrics in video generation.

CRePE supports external scene-geometry control and motion transfer.

Abstract

Camera-conditioned video generation requires positional encoding that remains reliable under changes in camera motion, lens configuration, and scene structure. However, existing attention-level camera encodings either provide ray-only camera signals or rely on pinhole camera geometry, limiting their applicability to general camera control under the Unified Camera Model, including wide-angle and fisheye lenses. To address this limitation, we propose Curved Ray Expectation Positional Encoding (CRePE). CRePE represents each image token as a depth-aware positional distribution along its source ray, providing a Unified Camera Model-compatible positional encoding that captures the projected-path geometry induced by wide-angle and fisheye cameras. CRePE is implemented through a Geometric Attention Adapter added to frozen video DiTs, injecting token-wise scene-distance information into selected attention layers and stabilizing it with pseudo supervision from a monocular geometry foundation model. This design leads to more stable camera control and improves several geometry-aware and perceptual-quality metrics, while remaining competitive on video-quality metrics. Controlled positional-encoding ablations show a better overall average rank than a RayRoPE-style endpoint PE baseline, demonstrating the effectiveness of UCM-aware projected-path integration across diverse camera models. Furthermore, by extending the same positional-encoding pathway to external geometry control through Radial MixForcing, CRePE supports external radial-map control for scene-geometry-conditioned generation and source-video motion transfer beyond camera control.