A Physical Coherence Benchmark for Evaluating Video Generation Models via Optical Flow-guided Frame Prediction

TL;DR

This paper introduces PhyCoBench, a benchmark for assessing physical coherence in video generation, along with PhyCoPredictor, an automated evaluation model that aligns well with human judgments.

Contribution

The paper presents a new benchmark and an automated evaluation model specifically designed to measure physical coherence in generated videos.

Findings

PhyCoPredictor closely matches human evaluations.

The benchmark covers 7 categories of physical principles.

The dataset and tools are publicly available on GitHub.

Abstract

Recent advances in video generation models demonstrate their potential as world simulators, but they often struggle with videos deviating from physical laws, a key concern overlooked by most text-to-video benchmarks. We introduce a benchmark designed specifically to assess the Physical Coherence of generated videos, PhyCoBench. Our benchmark includes 120 prompts covering 7 categories of physical principles, capturing key physical laws observable in video content. We evaluated four state-of-the-art (SoTA) T2V models on PhyCoBench and conducted manual assessments. Additionally, we propose an automated evaluation model: PhyCoPredictor, a diffusion model that generates optical flow and video frames in a cascade manner. Through a consistency evaluation comparing automated and manual sorting, the experimental results show that PhyCoPredictor currently aligns most closely with human evaluation. Therefore, it can effectively evaluate the physical coherence of videos, providing insights for future model optimization. Our benchmark, including physical coherence prompts, the automatic evaluation tool PhyCoPredictor, and the generated video dataset, has been released on GitHub at https://github.com/Jeckinchen/PhyCoBench.