Active World-Model with 4D-informed Retrieval for Exploration and Awareness

TL;DR

This paper introduces AW4RE, a generative world model that improves physical awareness and exploration in dynamic environments by integrating 4D-informed retrieval and action-conditioned predictions.

Contribution

It presents a novel awareness-centric model that combines 4D-informed evidence retrieval with generative completion to enhance exploration and sensing decision-making.

Findings

AW4RE produces more grounded predictions than geometry-aware baselines.

It handles extreme viewpoint shifts and temporal gaps effectively.

The model improves physical awareness in dynamic environments.

Abstract

Physical awareness, especially in a large and dynamic environment, is shaped by sensing decisions that determine observability across space, time, and scale, while observations impact the quality of sensing decisions. This loopy information structure makes physical awareness a fundamentally challenging decision problem with partial observations. While in the past decade we have witnessed the unprecedented success of reinforcement learning (RL) in problems with full observability, decision problems with partial observation, such as POMDPs, remain largely open: real-world explorations are excessively costly, while sim-to-real pipeline suffer from unobserved viewpoints. We introduce AW4RE (Active World-model with 4D-informed Retrieval for Exploration), an awareness-centric generative world model that provides a sensor-native surrogate environment for exploring sensing queries. Conditioned on a queried sensing action, AW4RE estimates the action-conditioned observation process. This is done by combining 4D-informed evidence retrieval, action-conditioned geometric support with temporal coherence, and conditional generative completion. Experiments demonstrate that AW4RE produces more grounded and consistent predictions than geometry-aware generative baselines under extreme viewpoint shifts, temporal gaps, and sparse geometric support.