Learning Interactive World Model for Object-Centric Reinforcement Learning

TL;DR

This paper presents FIOC-WM, a structured world model that learns object interactions explicitly, leading to more efficient and generalizable policies in object-centric reinforcement learning tasks.

Contribution

The paper introduces FIOC-WM, a novel framework that models object interactions explicitly within a structured world model for improved RL performance.

Findings

FIOC-WM enhances sample efficiency in policy learning.

FIOC-WM improves generalization across tasks.

Explicit interaction modeling is crucial for robust control.

Abstract

Agents that understand objects and their interactions can learn policies that are more robust and transferable. However, most object-centric RL methods factor state by individual objects while leaving interactions implicit. We introduce the Factored Interactive Object-Centric World Model (FIOC-WM), a unified framework that learns structured representations of both objects and their interactions within a world model. FIOC-WM captures environment dynamics with disentangled and modular representations of object interactions, improving sample efficiency and generalization for policy learning. Concretely, FIOC-WM first learns object-centric latents and an interaction structure directly from pixels, leveraging pre-trained vision encoders. The learned world model then decomposes tasks into composable interaction primitives, and a hierarchical policy is trained on top: a high level selects the type and order of interactions, while a low level executes them. On simulated robotic and embodied-AI benchmarks, FIOC-WM improves policy-learning sample efficiency and generalization over world-model baselines, indicating that explicit, modular interaction learning is crucial for robust control.