BiRP: Learning Robot Generalized Bimanual Coordination using Relative Parameterization Method on Human Demonstration

TL;DR

This paper introduces a novel relative parameterization method for learning and generalizing human-like bimanual coordination in robots, based on demonstration data, to improve complex manipulation tasks.

Contribution

It proposes a new Gaussian mixture model-based approach to learn and generalize two types of bimanual coordination from human demonstrations in robotics.

Findings

Successfully learned bimanual coordination from human data

Generalized coordination to new task parameters

Deployed on a humanoid robot for complex tasks

Abstract

Human bimanual manipulation can perform more complex tasks than a simple combination of two single arms, which is credited to the spatio-temporal coordination between the arms. However, the description of bimanual coordination is still an open topic in robotics. This makes it difficult to give an explainable coordination paradigm, let alone applied to robotics. In this work, we divide the main bimanual tasks in human daily activities into two types: leader-follower and synergistic coordination. Then we propose a relative parameterization method to learn these types of coordination from human demonstration. It represents coordination as Gaussian mixture models from bimanual demonstration to describe the change in the importance of coordination throughout the motions by probability. The learned coordinated representation can be generalized to new task parameters while ensuring spatio-temporal coordination. We demonstrate the method using synthetic motions and human demonstration data and deploy it to a humanoid robot to perform a generalized bimanual coordination motion. We believe that this easy-to-use bimanual learning from demonstration (LfD) method has the potential to be used as a data augmentation plugin for robot large manipulation model training. The corresponding codes are open-sourced in https://github.com/Skylark0924/Rofunc.