MetaWorld: Skill Transfer and Composition in a Hierarchical World Model for Grounding High-Level Instructions

TL;DR

MetaWorld introduces a hierarchical world model that enhances humanoid robot skill transfer and composition by integrating semantic planning with physical control, improving efficiency and generalization in complex tasks.

Contribution

The paper presents MetaWorld, a novel hierarchical framework that combines semantic and physical models, enabling efficient skill transfer and better task generalization for humanoid robots.

Findings

Outperforms existing world model-based RL in task completion

Achieves higher motion coherence in humanoid tasks

Enables efficient online adaptation through expert policy transfer

Abstract

Humanoid robot loco-manipulation remains constrained by the semantic-physical gap. Current methods face three limitations: Low sample efficiency in reinforcement learning, poor generalization in imitation learning, and physical inconsistency in VLMs. We propose MetaWorld, a hierarchical world model that integrates semantic planning and physical control via expert policy transfer. The framework decouples tasks into a VLM-driven semantic layer and a latent dynamics model operating in a compact state space. Our dynamic expert selection and motion prior fusion mechanism leverages a pre-trained multi-expert policy library as transferable knowledge, enabling efficient online adaptation via a two-stage framework. VLMs serve as semantic interfaces, mapping instructions to executable skills and bypassing symbol grounding. Experiments on Humanoid-Bench show MetaWorld outperforms world model-based RL in task completion and motion coherence. Our code will be found at https://anonymous.4open.science/r/metaworld-2BF4/