Adaptive World Models: Learning Behaviors by Latent Imagination Under Non-Stationarity

TL;DR

This paper introduces Hidden Parameter-POMDPs, a formalism for adaptive world models that learn robust, task-aware behaviors in non-stationary environments, advancing embodied intelligence.

Contribution

It presents a new formalism for control in non-stationary settings, enabling unsupervised learning of task abstractions and structured latent spaces.

Findings

Successfully learns robust behaviors in non-stationary RL benchmarks

Effectively learns task abstractions in an unsupervised manner

Creates structured, task-aware latent spaces

Abstract

Developing foundational world models is a key research direction for embodied intelligence, with the ability to adapt to non-stationary environments being a crucial criterion. In this work, we introduce a new formalism, Hidden Parameter-POMDP, designed for control with adaptive world models. We demonstrate that this approach enables learning robust behaviors across a variety of non-stationary RL benchmarks. Additionally, this formalism effectively learns task abstractions in an unsupervised manner, resulting in structured, task-aware latent spaces.