GrndCtrl: Grounding World Models via Self-Supervised Reward Alignment

TL;DR

This paper introduces GrndCtrl, a self-supervised framework that aligns pretrained video world models with physical and perceptual constraints to improve navigation stability and geometric grounding.

Contribution

It proposes RLWG, a novel post-training alignment method using multiple verifiable rewards, and instantiates it with GrndCtrl, enhancing spatial coherence in embodied navigation models.

Findings

GrndCtrl improves trajectory stability in navigation tasks.

The method achieves better geometric consistency than supervised fine-tuning.

It effectively bridges generative pretraining with grounded, reliable behavior.

Abstract

Recent advances in video world modeling have enabled large-scale generative models to simulate embodied environments with high visual fidelity, providing strong priors for prediction, planning, and control. Yet, despite their realism, these models often lack geometric grounding, limiting their use in navigation tasks that require spatial coherence and stability. We introduce Reinforcement Learning with World Grounding (RLWG), a self-supervised post-training framework that aligns pretrained world models with a physically verifiable structure through geometric and perceptual rewards. Analogous to reinforcement learning from verifiable feedback (RLVR) in language models, RLWG can use multiple rewards that measure pose cycle-consistency, depth reprojection, and temporal coherence. We instantiate this framework with GrndCtrl, a reward-aligned adaptation method based on Group Relative Policy Optimization (GRPO), yielding world models that maintain stable trajectories, consistent geometry, and reliable rollouts for embodied navigation. Like post-training alignment in large language models, GrndCtrl leverages verifiable rewards to bridge generative pretraining and grounded behavior, achieving superior spatial coherence and navigation stability over supervised fine-tuning in outdoor environments.