Slot-MPC: Goal-Conditioned Model Predictive Control with Object-Centric Representations

TL;DR

Slot-MPC introduces an object-centric world modeling framework that uses differentiable, slot-based representations for efficient, goal-conditioned planning via Model Predictive Control in robotic tasks.

Contribution

It presents a novel, differentiable, object-centric world model enabling gradient-based MPC for improved planning and generalization in robotic manipulation.

Findings

Slot-MPC outperforms non-object-centric baselines in task success.

Gradient-based MPC is more efficient and effective than sampling-based methods.

Explicit object-centric representations enhance generalization to unseen scenarios.

Abstract

Predictive world models enable agents to model scene dynamics and reason about the consequences of their actions. Inspired by human perception, object-centric world models capture scene dynamics using object-level representations, which can be used for downstream applications such as action planning. However, most object-centric world models and reinforcement learning (RL) approaches learn reactive policies that are fixed at inference time, limiting generalization to novel situations. We propose Slot-MPC, an object-centric world modeling framework that enables planning through Model Predictive Control (MPC). Slot-MPC leverages vision encoders to learn slot-based representations, which encode individual objects in the scene, and uses these structured representations to learn an action-conditioned object-centric dynamics model. At inference time, the learned dynamics model enables action planning via MPC, allowing agents to adapt to previously unseen situations. Since the learned world model is differentiable, we can use gradient-based MPC to directly optimize actions, which is computationally more efficient than relying on gradient-free, sampling-based MPC methods. Experiments on simulated robotic manipulation tasks show that Slot-MPC improves both task performance and planning efficiency compared to non-object-centric world model baselines. In the considered offline setting with limited state-action coverage, we find that gradient-based MPC performs better than gradient-free, sampling-based MPC. Our results demonstrate that explicitly structured, object-centric representations provide a strong inductive bias for controllable and generalizable decision-making. Code and additional results are available at https://slot-mpc.github.io.