Demonstrate Once, Imitate Immediately (DOME): Learning Visual Servoing for One-Shot Imitation Learning

TL;DR

DOME is a one-shot imitation learning method that enables robots to learn and perform tasks from a single demonstration without additional training, using visual segmentation and servoing.

Contribution

DOME introduces a novel approach combining visual segmentation and servoing for immediate task execution from one demonstration, without prior knowledge.

Findings

Achieves near 100% success on 7 real-world tasks

Operates without prior task or object knowledge

Handles novel object configurations and distractors

Abstract

We present DOME, a novel method for one-shot imitation learning, where a task can be learned from just a single demonstration and then be deployed immediately, without any further data collection or training. DOME does not require prior task or object knowledge, and can perform the task in novel object configurations and with distractors. At its core, DOME uses an image-conditioned object segmentation network followed by a learned visual servoing network, to move the robot's end-effector to the same relative pose to the object as during the demonstration, after which the task can be completed by replaying the demonstration's end-effector velocities. We show that DOME achieves near 100% success rate on 7 real-world everyday tasks, and we perform several studies to thoroughly understand each individual component of DOME. Videos and supplementary material are available at: https://www.robot-learning.uk/dome .