Hierarchical Planning with Latent World Models

TL;DR

This paper introduces a hierarchical planning method with multi-scale latent world models that improves long-horizon control and zero-shot generalization in embodied tasks, reducing planning complexity and increasing success rates.

Contribution

It proposes a novel hierarchical approach that learns latent world models at multiple temporal scales, enabling efficient long-horizon reasoning across diverse domains.

Findings

Achieves 70% success rate on real-world pick-and-place tasks with only goal specification.

Outperforms single-level models in success rate and planning efficiency.

Requires up to 4x less planning time in simulated environments.

Abstract

Model predictive control (MPC) with learned world models has emerged as a promising paradigm for embodied control, particularly for its ability to generalize zero-shot when deployed in new environments. However, learned world models often struggle with long-horizon control due to the accumulation of prediction errors and the exponentially growing search space. In this work, we address these challenges by learning latent world models at multiple temporal scales and performing hierarchical planning across these scales, enabling long-horizon reasoning while substantially reducing inference-time planning complexity. Our approach serves as a modular planning abstraction that applies across diverse latent world-model architectures and domains. We demonstrate that this hierarchical approach enables zero-shot control on real-world non-greedy robotic tasks, achieving a 70% success rate on pick-&-place using only a final goal specification, compared to 0% for a single-level world model. In addition, across physics-based simulated environments including push manipulation and maze navigation, hierarchical planning achieves higher success while requiring up to 4x less planning-time compute.