PhysVideo: Physically Plausible Video Generation with Cross-View Geometry Guidance

TL;DR

PhysVideo introduces a two-stage framework for physically plausible video generation that incorporates physics-aware attention and geometry-guided synthesis, significantly enhancing realism and coherence in generated videos.

Contribution

The paper presents PhysVideo, a novel two-stage approach with physics-aware and geometry-enhanced modules, and introduces PhysMV, a large dataset for training physically consistent video models.

Findings

PhysVideo outperforms existing methods in physical realism.

The framework achieves higher spatial-temporal coherence.

Extensive experiments validate the effectiveness of PhysVideo.

Abstract

Recent progress in video generation has led to substantial improvements in visual fidelity, yet ensuring physically consistent motion remains a fundamental challenge. Intuitively, this limitation can be attributed to the fact that real-world object motion unfolds in three-dimensional space, while video observations provide only partial, view-dependent projections of such dynamics. To address these issues, we propose PhysVideo, a two-stage framework that first generates physics-aware orthogonal foreground videos and then synthesizes full videos with background. In the first stage, Phys4View leverages physics-aware attention to capture the influence of physical attributes on motion dynamics, and enhances spatio-temporal consistency by incorporating geometry-enhanced cross-view attention and temporal attention. In the second stage, VideoSyn uses the generated foreground videos as guidance and learns the interactions between foreground dynamics and background context for controllable video synthesis. To support training, we construct PhysMV, a dataset containing 40K scenes, each consisting of four orthogonal viewpoints, resulting in a total of 160K video sequences. Extensive experiments demonstrate that PhysVideo significantly improves physical realism and spatial-temporal coherence over existing video generation methods. Home page: https://anonymous.4open.science/w/Phys4D/.