G-DReaM: Graph-conditioned Diffusion Retargeting across Multiple Embodiments

TL;DR

This paper introduces a unified graph-conditioned diffusion framework for motion retargeting across diverse robot embodiments, effectively capturing topological and geometrical features to transfer motions between heterogeneous structures.

Contribution

It presents a novel graph-based encoding and attention mechanism for cross-embodiment motion retargeting, addressing topological and geometrical inconsistencies in a unified model.

Findings

Successfully retargets motions across different robot structures

Demonstrates generalization to various skeletal structures

Validates effectiveness through experimental results

Abstract

Motion retargeting for specific robot from existing motion datasets is one critical step in transferring motion patterns from human behaviors to and across various robots. However, inconsistencies in topological structure, geometrical parameters as well as joint correspondence make it difficult to handle diverse embodiments with a unified retargeting architecture. In this work, we propose a novel unified graph-conditioned diffusion-based motion generation framework for retargeting reference motions across diverse embodiments. The intrinsic characteristics of heterogeneous embodiments are represented with graph structure that effectively captures topological and geometrical features of different robots. Such a graph-based encoding further allows for knowledge exploitation at the joint level with a customized attention mechanisms developed in this work. For lacking ground truth motions of the desired embodiment, we utilize an energy-based guidance formulated as retargeting losses to train the diffusion model. As one of the first cross-embodiment motion retargeting methods in robotics, our experiments validate that the proposed model can retarget motions across heterogeneous embodiments in a unified manner. Moreover, it demonstrates a certain degree of generalization to both diverse skeletal structures and similar motion patterns.