Envision: Benchmarking Unified Understanding & Generation for Causal World Process Insights

TL;DR

Envision introduces a benchmark and metric for evaluating multimodal models on their ability to generate and understand dynamic, causally-structured visual narratives over time, addressing static pattern matching limitations.

Contribution

It proposes Envision, a causal event progression benchmark with a new holistic metric, enabling evaluation of models' understanding of spatiotemporal causality in multi-image generation.

Findings

Unified models outperform specialized T2I models in causal coherence

Specialized T2I models excel in aesthetics but lack world knowledge

Even advanced models struggle with spatiotemporal consistency

Abstract

Current multimodal models aim to transcend the limitations of single-modality representations by unifying understanding and generation, often using text-to-image (T2I) tasks to calibrate semantic consistency. However, their reliance on static, single-image generation in training and evaluation leads to overfitting to static pattern matching and semantic fusion, while fundamentally hindering their ability to model dynamic processes that unfold over time. To address these constraints, we propose Envision-a causal event progression benchmark for chained text-to-multi-image generation. Grounded in world knowledge and structured by spatiotemporal causality, it reorganizes existing evaluation dimensions and includes 1,000 four-stage prompts spanning six scientific and humanities domains. To transition evaluation from single images to sequential frames and assess whether models truly internalize world knowledge while adhering to causal-temporal constraints, we introduce Envision-Score, a holistic metric integrating multi-dimensional consistency, physicality, and aesthetics. Comprehensive evaluation of 15 models (10 specialized T2I models, 5 unified models) uncovers: specialized T2I models demonstrate proficiency in aesthetic rendering yet lack intrinsic world knowledge. Unified multimodal models bridge this gap, consistently outperforming specialized counterparts in causal narrative coherence. However, even these unified architectures remain subordinate to closed-source models and struggle to overcome the core challenge of spatiotemporal consistency. This demonstrates that a focus on causally-isolated single images impedes multi-frame reasoning and generation, promoting static pattern matching over dynamic world modeling-ultimately limiting world knowledge internalization, generation.