DemoDiffusion: One-Shot Human Imitation using pre-trained Diffusion Policy

TL;DR

DemoDiffusion enables robots to imitate a single human demonstration for manipulation tasks by combining kinematic retargeting with a pre-trained diffusion policy, achieving high success rates without task-specific training.

Contribution

It introduces DemoDiffusion, a novel approach that integrates human motion priors with pre-trained diffusion policies for one-shot robot imitation.

Findings

Achieves 83.8% success rate across 8 tasks

Outperforms pure retargeting and pre-trained policy alone

Demonstrates robustness in diverse manipulation scenarios

Abstract

We propose DemoDiffusion, a simple method for enabling robots to perform manipulation tasks by imitating a single human demonstration, without requiring task-specific training or paired human-robot data. Our approach is based on two insights. First, the hand motion in a human demonstration provides a useful prior for the robot's end-effector trajectory, which we can convert into a rough open-loop robot motion trajectory via kinematic retargeting. Second, while this retargeted motion captures the overall structure of the task, it may not align well with plausible robot actions in-context. To address this, we leverage a pre-trained generalist diffusion policy to modify the trajectory, ensuring it both follows the human motion and remains within the distribution of plausible robot actions. Unlike approaches based on online reinforcement learning or paired human-robot data, our method enables robust adaptation to new tasks and scenes with minimal effort. In real-world experiments across 8 diverse manipulation tasks, DemoDiffusion achieves 83.8\% average success rate, compared to 13.8\% for the pre-trained policy and 52.5\% for kinematic retargeting, succeeding even on tasks where the pre-trained generalist policy fails entirely. Project page: https://demodiffusion.github.io/