CKT-WAM: Parameter-Efficient Context Knowledge Transfer Between World Action Models

TL;DR

CKT-WAM introduces a parameter-efficient framework for transferring knowledge between heterogeneous world action models by embedding context in text space, enhancing zero-shot generalization and real-world manipulation.

Contribution

It proposes a novel context knowledge transfer method that reduces adaptation cost and improves performance with minimal parameter updates.

Findings

Achieves 86.1% success rate on LIBERO-Plus with only 1.17% trainable parameters.

Demonstrates strong real-world long-horizon manipulation with 83.3% success rate.

Outperforms existing methods in zero-shot generalization and multi-task settings.

Abstract

World action models (WAMs) provide a powerful generative framework for embodied control, yet transferring knowledge across heterogeneous WAMs remains challenging due to mismatched latent interfaces, high adaptation cost, and the rigidity of conventional distillation objectives. We propose \textbf{CKT-WAM}, a parameter-efficient \textbf{C}ontext \textbf{K}nowledge \textbf{T}ransfer framework that transfers teacher WAM's knowledge into a student WAM through a compact context in the text embedding space, rather than output imitation or dense hidden-state matching. Specifically, CKT-WAM extracts intermediate teacher hidden states, reduces the number of tokens via compressors' learnable-query cross attention (LQCA), and transforms them through an always-on generalized adapter, a lightweight router, and sparsely activated specialized adapters. The resulting context is then appended to the student's conditioning textual embeddings, thereby injecting the transferred knowledge into the student with minimal architectural modification. Experiments show that CKT-WAM consistently improves zero-shot generalization and achieves the best overall performance on LIBERO-Plus, reaching 86.1\% total success rate with only 1.17\% trainable parameters, while approaching full fine-tuning performance. Beyond simulation, CKT-WAM also demonstrates strong real-world long-horizon manipulation ability, achieving the best average success rate of 83.3\% across four multi-step and long-horizon tasks. Code is available at https://github.com/YuhuaJiang2002/CKT-WAM.