Multi-Domain Motion Embedding: Expressive Real-Time Mimicry for Legged Robots

TL;DR

This paper introduces Multi-Domain Motion Embedding (MDME), a novel motion representation that captures both structured and irregular motion patterns, enabling real-time, accurate, and generalizable robot imitation across diverse styles and morphologies.

Contribution

We propose MDME, a wavelet-based and probabilistic motion embedding that unifies structured and unstructured features for improved real-time robot motion imitation.

Findings

MDME outperforms prior methods in motion reconstruction fidelity.

MDME generalizes well to unseen motions and styles.

MDME enables zero-shot real-time motion style reproduction.

Abstract

Effective motion representation is crucial for enabling robots to imitate expressive behaviors in real time, yet existing motion controllers often ignore inherent patterns in motion. Previous efforts in representation learning do not attempt to jointly capture structured periodic patterns and irregular variations in human and animal movement. To address this, we present Multi-Domain Motion Embedding (MDME), a motion representation that unifies the embedding of structured and unstructured features using a wavelet-based encoder and a probabilistic embedding in parallel. This produces a rich representation of reference motions from a minimal input set, enabling improved generalization across diverse motion styles and morphologies. We evaluate MDME on retargeting-free real-time motion imitation by conditioning robot control policies on the learned embeddings, demonstrating accurate reproduction of complex trajectories on both humanoid and quadruped platforms. Our comparative studies confirm that MDME outperforms prior approaches in reconstruction fidelity and generalizability to unseen motions. Furthermore, we demonstrate that MDME can reproduce novel motion styles in real-time through zero-shot deployment, eliminating the need for task-specific tuning or online retargeting. These results position MDME as a generalizable and structure-aware foundation for scalable real-time robot imitation.