FUNCTO: Function-Centric One-Shot Imitation Learning for Tool Manipulation

TL;DR

FUNCTO introduces a function-centric one-shot imitation learning approach that enables robots to generalize tool manipulation skills from a single demonstration to diverse tools with the same function, despite geometric variations.

Contribution

The paper proposes a novel method, FUNCTO, that uses 3D functional keypoints to establish correspondences and generalize tool use skills across different tools with the same function.

Findings

FUNCTO outperforms existing OSIL methods in generalizing to new tools.

Real-robot experiments validate the effectiveness of FUNCTO in diverse tasks.

The method handles significant intra-function geometric variations successfully.

Abstract

Learning tool use from a single human demonstration video offers a highly intuitive and efficient approach to robot teaching. While humans can effortlessly generalize a demonstrated tool manipulation skill to diverse tools that support the same function (e.g., pouring with a mug versus a teapot), current one-shot imitation learning (OSIL) methods struggle to achieve this. A key challenge lies in establishing functional correspondences between demonstration and test tools, considering significant geometric variations among tools with the same function (i.e., intra-function variations). To address this challenge, we propose FUNCTO (Function-Centric OSIL for Tool Manipulation), an OSIL method that establishes function-centric correspondences with a 3D functional keypoint representation, enabling robots to generalize tool manipulation skills from a single human demonstration video to novel tools with the same function despite significant intra-function variations. With this formulation, we factorize FUNCTO into three stages: (1) functional keypoint extraction, (2) function-centric correspondence establishment, and (3) functional keypoint-based action planning. We evaluate FUNCTO against exiting modular OSIL methods and end-to-end behavioral cloning methods through real-robot experiments on diverse tool manipulation tasks. The results demonstrate the superiority of FUNCTO when generalizing to novel tools with intra-function geometric variations. More details are available at https://sites.google.com/view/functo.