Learning Forceful Manipulation Skills from Multi-modal Human Demonstrations

TL;DR

This paper extends Learning from Demonstration to forceful manipulation skills by integrating multi-modal demonstrations, enabling robots to perform complex industrial tasks involving force and pose profiles with scene adaptation.

Contribution

It introduces a novel framework combining task-parameterized optimization and impedance control for forceful skills, including pose and force features, with an online adaptation algorithm.

Findings

Validated on a 7-DoF robot for E-bike assembly tasks

Successfully performed insertion, sliding, and twisting operations

Demonstrated reliable reproduction of forceful manipulation skills

Abstract

Learning from Demonstration (LfD) provides an intuitive and fast approach to program robotic manipulators. Task parameterized representations allow easy adaptation to new scenes and online observations. However, this approach has been limited to pose-only demonstrations and thus only skills with spatial and temporal features. In this work, we extend the LfD framework to address forceful manipulation skills, which are of great importance for industrial processes such as assembly. For such skills, multi-modal demonstrations including robot end-effector poses, force and torque readings, and operation scene are essential. Our objective is to reproduce such skills reliably according to the demonstrated pose and force profiles within different scenes. The proposed method combines our previous work on task-parameterized optimization and attractor-based impedance control. The learned skill model consists of (i) the attractor model that unifies the pose and force features, and (ii) the stiffness model that optimizes the stiffness for different stages of the skill. Furthermore, an online execution algorithm is proposed to adapt the skill execution to real-time observations of robot poses, measured forces, and changed scenes. We validate this method rigorously on a 7-DoF robot arm over several steps of an E-bike motor assembly process, which require different types of forceful interaction such as insertion, sliding and twisting.