Foundation World Models for Agents that Learn, Verify, and Adapt Reliably Beyond Static Environments

TL;DR

This paper proposes foundation world models that enable autonomous agents to learn, verify, and adapt reliably in dynamic, open environments by integrating reinforcement learning, formal verification, and abstraction mechanisms.

Contribution

It introduces a comprehensive framework combining learnable reward models, adaptive verification, and online abstraction to support reliable, adaptable, and verifiable agent behavior.

Findings

Framework supports synthesis of verifiable programs

Agents can derive policies from limited interactions

Models maintain correctness while adapting to new conditions

Abstract

The next generation of autonomous agents must not only learn efficiently but also act reliably and adapt their behavior in open worlds. Standard approaches typically assume fixed tasks and environments with little or no novelty, which limits world models' ability to support agents that must evolve their policies as conditions change. This paper outlines a vision for foundation world models: persistent, compositional representations that unify reinforcement learning, reactive/program synthesis, and abstraction mechanisms. We propose an agenda built around four components: (i) learnable reward models from specifications to support optimization with clear objectives; (ii) adaptive formal verification integrated throughout learning; (iii) online abstraction calibration to quantify the reliability of the model's predictions; and (iv) test-time synthesis and world-model generation guided by verifiers. Together, these components enable agents to synthesize verifiable programs, derive new policies from a small number of interactions, and maintain correctness while adapting to novelty. The resulting framework positions foundation world models as a substrate for learning, reasoning, and adaptation, laying the groundwork for agents that not only act well but can explain and justify the behavior they adopt.