Epipolar Geometry Improves Video Generation Models

TL;DR

This paper introduces a method that incorporates epipolar geometry constraints into video diffusion models, significantly improving their geometric consistency and stability without sacrificing visual quality.

Contribution

It presents a novel approach that aligns diffusion models with classical geometric principles using preference-based optimization, enhancing 3D consistency in video generation.

Findings

Classical geometric constraints outperform learned metrics in stability.

The method generalizes well from static scenes to dynamic content.

Geometric enforcement improves visual and spatial consistency.

Abstract

Video generation models have progressed tremendously through large latent diffusion transformers trained with rectified flow techniques. Yet these models still struggle with geometric inconsistencies, unstable motion, and visual artifacts that break the illusion of realistic 3D scenes. 3D-consistent video generation could significantly impact numerous downstream applications in generation and reconstruction tasks. We explore how epipolar geometry constraints improve modern video diffusion models. Despite massive training data, these models fail to capture fundamental geometric principles underlying visual content. We align diffusion models using pairwise epipolar geometry constraints via preference-based optimization, directly addressing unstable camera trajectories and geometric artifacts through mathematically principled geometric enforcement. Our approach efficiently enforces geometric principles without requiring end-to-end differentiability. Evaluation demonstrates that classical geometric constraints provide more stable optimization signals than modern learned metrics, which produce noisy targets that compromise alignment quality. Training on static scenes with dynamic cameras ensures high-quality measurements while the model generalizes effectively to diverse dynamic content. By bridging data-driven deep learning with classical geometric computer vision, we present a practical method for generating spatially consistent videos without compromising visual quality.