A Sensorimotor Reinforcement Learning Framework for Physical Human-Robot Interaction

TL;DR

This paper introduces a data-efficient reinforcement learning framework enabling robots to learn collaborative tasks with humans by modeling human behavior uncertainty with Gaussian processes and optimizing actions through Bayesian methods.

Contribution

The paper presents a novel sensorimotor reinforcement learning approach that efficiently learns human-robot collaboration tasks using Gaussian processes and Bayesian optimization.

Findings

Fast and data-efficient model learning demonstrated

Effective handling of human action uncertainty

Successful joint control of a ball on a plank with a PR2 robot

Abstract

Modeling of physical human-robot collaborations is generally a challenging problem due to the unpredictive nature of human behavior. To address this issue, we present a data-efficient reinforcement learning framework which enables a robot to learn how to collaborate with a human partner. The robot learns the task from its own sensorimotor experiences in an unsupervised manner. The uncertainty of the human actions is modeled using Gaussian processes (GP) to implement action-value functions. Optimal action selection given the uncertain GP model is ensured by Bayesian optimization. We apply the framework to a scenario in which a human and a PR2 robot jointly control the ball position on a plank based on vision and force/torque data. Our experimental results show the suitability of the proposed method in terms of fast and data-efficient model learning, optimal action selection under uncertainties and equal role sharing between the partners.