Remote Task-oriented Grasp Area Teaching By Non-Experts through Interactive Segmentation and Few-Shot Learning

TL;DR

This paper introduces a two-step framework enabling non-experts to teach robots task-specific grasping through interactive segmentation and few-shot learning, improving robot adaptability in unstructured environments.

Contribution

The work presents a novel combination of interactive segmentation, a new grasp area segmentation dataset, and meta-learning for effective few-shot grasp area estimation.

Findings

Successfully detects correct grasp areas in unseen scenes.

Enables remote teaching of new grasp strategies by non-experts.

Uses a novel dataset and meta-learning for few-shot adaptation.

Abstract

A robot operating in unstructured environments must be able to discriminate between different grasping styles depending on the prospective manipulation task. Having a system that allows learning from remote non-expert demonstrations can very feasibly extend the cognitive skills of a robot for task-oriented grasping. We propose a novel two-step framework towards this aim. The first step involves grasp area estimation by segmentation. We receive grasp area demonstrations for a new task via interactive segmentation, and learn from these few demonstrations to estimate the required grasp area on an unseen scene for the given task. The second step is autonomous grasp estimation in the segmented region. To train the segmentation network for few-shot learning, we built a grasp area segmentation (GAS) dataset with 10089 images grouped into 1121 segmentation tasks. We benefit from an efficient meta learning algorithm for training for few-shot adaptation. Experimental evaluation showed that our method successfully detects the correct grasp area on the respective objects in unseen test scenes and effectively allows remote teaching of new grasp strategies by non-experts.