Feedback World Model Enables Precise Guidance of Diffusion Policy

TL;DR

The paper introduces a feedback world model that uses real-time observations to iteratively correct predictions during inference, significantly enhancing robotic decision-making accuracy and robustness under distribution shifts.

Contribution

It proposes a novel feedback mechanism for world models that updates predictions online without additional training, improving performance in out-of-distribution tasks.

Findings

Reduces world model prediction error by up to 76.4%.

Improves out-of-distribution success rate by 30%.

Enhances policy performance in real-world manipulation tasks.

Abstract

World models aim to improve robotic decision making by predicting the consequences of actions. However, in practice, their predictions often become unreliable once the robot encounters states outside the training distribution, limiting their effectiveness at deployment. We observe that execution itself provides a natural but underutilized signal: after each action, the robot directly observes the true next state, revealing the mismatch between predicted and actual outcomes. Building on this insight, we propose feedback world model, a new paradigm that closes the loop between prediction and observation at inference time. Instead of treating the world model as a static open-loop predictor, our method maintains a lightweight feedback state that is updated online to iteratively correct future predictions, compensating for model errors using real-time observations without additional training data or parameter updates. We show that this process can be interpreted as a latent-space observer and admits convergence guarantees under mild conditions. We further introduce action-aware guidance to better translate corrected predictions into control by emphasizing action-controllable components while suppressing irrelevant variations. Experiments on LIBERO-Plus, Robomimic, and real-world manipulation tasks demonstrate that our method substantially improves both prediction accuracy and policy performance under distribution shift. In particular, it reduces world model prediction error by up to 76.4% and improves out-of-distribution (OOD) success rate by 30%. These results show that incorporating real-time feedback at inference time provides a simple yet powerful alternative to static world modeling.