Demo-JEPA: Joint-Embedding Predictive Architecture for One-shot Cross-Embodiment Imitation

TL;DR

Demo-JEPA introduces a cross-embodiment imitation framework that infers demonstrator goals from visual cues, enabling flexible, embodiment-agnostic imitation in robotics.

Contribution

It presents a novel approach that decouples demonstration intent from embodiment, allowing imitation across different morphologies without shared action spaces.

Findings

Matches specialized in-domain planners in experiments.

Generalizes to unseen tasks and embodiments.

Requires only visual demonstrations and own interaction experience.

Abstract

Robotic imitation learning is often treated as reproducing demonstrated actions, but actions are inherently embodiment-specific. When demonstrations come from humans or robots with different morphology, kinematics, or action spaces, this action-centric view requires shared action spaces, heuristic retargeting, or large-scale multi-embodiment co-training. We instead view demonstrations as implicit specifications of future goals: the target agent should infer what state the demonstrator is trying to realize, rather than how the demonstrator executes it. We propose Demo-JEPA, a cross-embodiment imitation framework that decouples demonstration intent from embodiment-specific execution. Built on a JEPA-based world model, Demo-JEPA translates source visual demonstrations into target-compatible future latent trajectories in a shared predictive representation space. The target agent then uses these latent trajectories as subgoals and realizes them through planning under its own learned forward dynamics. Because Demo-JEPA avoids action-level correspondence and requires only visual demonstrations plus the target agent's own interaction experience, it supports flexible imitation across heterogeneous embodiments. Experiments on RLBench and real-world manipulation tasks show that Demo-JEPA matches specialized in-domain planners and generalizes to unseen tasks and embodiment configurations where prior methods fail.