Uncertainty-aware Contact-safe Model-based Reinforcement Learning

TL;DR

This paper introduces a contact-safe model-based reinforcement learning method that adjusts control limits based on model uncertainty to prevent damage during learning in contact-rich robotic tasks.

Contribution

It proposes an uncertainty-aware control scheme that integrates probabilistic model predictive control with model uncertainty to enhance safety during robot learning.

Findings

Effective in simulated bowl mixing tasks

Successful in real robot scooping tasks

Reduces risk of damage during learning

Abstract

This letter presents contact-safe Model-based Reinforcement Learning (MBRL) for robot applications that achieves contact-safe behaviors in the learning process. In typical MBRL, we cannot expect the data-driven model to generate accurate and reliable policies to the intended robotic tasks during the learning process due to sample scarcity. Operating these unreliable policies in a contact-rich environment could cause damage to the robot and its surroundings. To alleviate the risk of causing damage through unexpected intensive physical contacts, we present the contact-safe MBRL that associates the probabilistic Model Predictive Control's (pMPC) control limits with the model uncertainty so that the allowed acceleration of controlled behavior is adjusted according to learning progress. Control planning with such uncertainty-aware control limits is formulated as a deterministic MPC problem using a computation-efficient approximated GP dynamics and an approximated inference technique. Our approach's effectiveness is evaluated through bowl mixing tasks with simulated and real robots, scooping tasks with a real robot as examples of contact-rich manipulation skills. (video: https://youtu.be/sdhHP3NhYi0)