Infrastructure-Centric World Models: Bridging Temporal Depth and Spatial Breadth for Roadside Perception

TL;DR

This paper proposes infrastructure-centric world models for autonomous driving, leveraging roadside sensors for long-term behavioral understanding and vehicle sensors for spatial scene sampling, aiming to enhance traffic perception and prediction.

Contribution

It introduces a novel three-phase framework for infrastructure-centric world models, integrating multi-sensor data, uncertainty propagation, and collaborative V2X communication.

Findings

Proposes a dual-layer architecture for perception and world modeling.

Defines a taxonomy of driving world model paradigms and positions I-WM within it.

Identifies open-source foundations for each phase of the proposed framework.

Abstract

World models, generative AI systems that simulate how environments evolve, are transforming autonomous driving, yet all existing approaches adopt an ego-vehicle perspective, leaving the infrastructure viewpoint unexplored. We argue that infrastructure-centric world models offer a fundamentally complementary capability: the bird's-eye, multi-sensor, persistent viewpoint that roadside systems uniquely possess. Central to our thesis is a spatio-temporal complementarity: fixed roadside sensors excel at temporal depth, accumulating long-term behavioral distributions including rare safety-critical events, while vehicle-borne sensors excel at spatial breadth, sampling diverse scenes across large road networks. This paper presents a vision for Infrastructure-centric World Models (I-WM) in three phases: (I) generative scene understanding with quality-aware uncertainty propagation, (II) physics-informed predictive dynamics with multi-agent counterfactual reasoning, and (III) collaborative world models for V2X communication via latent space alignment. We propose a dual-layer architecture, annotation-free perception as a multi-modal data engine feeding end-to-end generative world models, with a phased sensor strategy from LiDAR through 4D radar and signal phase data to event cameras. We establish a taxonomy of driving world model paradigms, position I-WM relative to LeCun's JEPA, Li Fei-Fei's spatial intelligence, and VLA architectures, and introduce Infrastructure VLA (I-VLA) as a novel unification of roadside perception, language commands, and traffic control actions. Our vision builds upon existing multi-LiDAR pipelines and identifies open-source foundations for each phase, providing a path toward infrastructure that understands and anticipates traffic.