World Action Verifier: Self-Improving World Models via Forward-Inverse Asymmetry

TL;DR

The paper introduces WAV, a framework that enhances world models by enabling them to self-verify and improve through decomposing predictions into plausibility and reachability, leveraging asymmetries in data.

Contribution

WAV presents a novel self-verification approach for world models using cycle consistency and asymmetry exploitation, improving robustness and sample efficiency.

Findings

Achieves 2x higher sample efficiency across nine tasks.

Improves downstream policy performance by 18%.

Effectively verifies predictions in under-explored regimes.

Abstract

General-purpose world models promise scalable policy evaluation, optimization, and planning, yet achieving the required level of robustness remains challenging. Unlike policy learning, which primarily focuses on optimal actions, a world model must be reliable over a much broader range of suboptimal actions, which are often insufficiently covered by action-labeled interaction data. To address this challenge, we propose World Action Verifier (WAV), a framework that enables world models to identify their own prediction errors and self-improve. The key idea is to decompose action-conditioned state prediction into two factors -- state plausibility and action reachability -- and verify each separately. We show that these verification problems can be substantially easier than predicting future states due to two underlying asymmetries: the broader availability of action-free data and the lower dimensionality of action-relevant features. Leveraging these asymmetries, we augment a world model with (i) a diverse subgoal generator obtained from video corpora and (ii) a sparse inverse model that infers actions from a subset of state features. By enforcing cycle consistency among generated subgoals, inferred actions, and forward rollouts, WAV provides an effective verification mechanism in under-explored regimes, where existing methods typically fail. Across nine tasks spanning MiniGrid, RoboMimic, and ManiSkill, our method achieves 2x higher sample efficiency while improving downstream policy performance by 18%.