PhysAlign: Physics-Coherent Image-to-Video Generation through Feature and 3D Representation Alignment

TL;DR

PhysAlign introduces a physics-coherent image-to-video generation framework that uses synthetic data and explicit 3D constraints to produce temporally stable videos aligned with physical laws.

Contribution

It presents a novel physics-grounded approach for video generation, utilizing a synthetic dataset and a unified physical latent space to improve temporal coherence.

Findings

Outperforms existing models in physical reasoning tasks

Achieves higher temporal stability without sacrificing visual quality

Bridges the gap between visual synthesis and physical kinematics

Abstract

Video Diffusion Models (VDMs) offer a promising approach for simulating dynamic scenes and environments, with broad applications in robotics and media generation. However, existing models often generate temporally incoherent content that violates basic physical intuition, significantly limiting their practical applicability. We propose PhysAlign, an efficient framework for physics-coherent image-to-video (I2V) generation that explicitly addresses this limitation. To overcome the critical scarcity of physics-annotated videos, we first construct a fully controllable synthetic data generation pipeline based on rigid-body simulation, yielding a highly-curated dataset with accurate, fine-grained physics and 3D annotations. Leveraging this data, PhysAlign constructs a unified physical latent space by coupling explicit 3D geometry constraints with a Gram-based spatio-temporal relational alignment that extracts kinematic priors from video foundation models. Extensive experiments demonstrate that PhysAlign significantly outperforms existing VDMs on tasks requiring complex physical reasoning and temporal stability, without compromising zero-shot visual quality. PhysAlign shows the potential to bridge the gap between raw visual synthesis and rigid-body kinematics, establishing a practical paradigm for genuinely physics-grounded video generation. The project page is available at https://physalign.github.io/PhysAlign.