GeoVideo: Introducing Geometric Regularization into Video Generation Model

TL;DR

This paper enhances video generation models by integrating geometric regularization through depth prediction, significantly improving temporal consistency and 3D structural coherence in synthesized videos.

Contribution

It introduces a novel geometric regularization framework using depth prediction and multi-view loss to enforce 3D structural consistency in diffusion-based video generation.

Findings

Improved temporal stability and geometric consistency in generated videos.

Enhanced shape and structure coherence across frames.

Significant performance gains over baseline models.

Abstract

Recent advances in video generation have enabled the synthesis of high-quality and visually realistic clips using diffusion transformer models. However, most existing approaches operate purely in the 2D pixel space and lack explicit mechanisms for modeling 3D structures, often resulting in temporally inconsistent geometries, implausible motions, and structural artifacts. In this work, we introduce geometric regularization losses into video generation by augmenting latent diffusion models with per-frame depth prediction. We adopted depth as the geometric representation because of the great progress in depth prediction and its compatibility with image-based latent encoders. Specifically, to enforce structural consistency over time, we propose a multi-view geometric loss that aligns the predicted depth maps across frames within a shared 3D coordinate system. Our method bridges the gap between appearance generation and 3D structure modeling, leading to improved spatio-temporal coherence, shape consistency, and physical plausibility. Experiments across multiple datasets show that our approach produces significantly more stable and geometrically consistent results than existing baselines.