Efficient Recovery Learning using Model Predictive Meta-Reasoning

TL;DR

This paper introduces MetaReSkill, an online meta-reasoning algorithm that efficiently learns recovery skills for manipulation tasks, significantly improving success rates in simulation and real-world robot experiments.

Contribution

The paper presents MetaReSkill, a novel online algorithm for sample-efficient recovery learning that dynamically allocates training resources based on potential task improvements.

Findings

Recovery skills increased success rates from 71% to 92.4% in simulation.

Recovery skills increased success rates from 75% to 90% on a real robot.

MetaReSkill effectively identifies and improves critical recovery policies.

Abstract

Operating under real world conditions is challenging due to the possibility of a wide range of failures induced by execution errors and state uncertainty. In relatively benign settings, such failures can be overcome by retrying or executing one of a small number of hand-engineered recovery strategies. By contrast, contact-rich sequential manipulation tasks, like opening doors and assembling furniture, are not amenable to exhaustive hand-engineering. To address this issue, we present a general approach for robustifying manipulation strategies in a sample-efficient manner. Our approach incrementally improves robustness by first discovering the failure modes of the current strategy via exploration in simulation and then learning additional recovery skills to handle these failures. To ensure efficient learning, we propose an online algorithm called Meta-Reasoning for Skill Learning (MetaReSkill) that monitors the progress of all recovery policies during training and allocates training resources to recoveries that are likely to improve the task performance the most. We use our approach to learn recovery skills for door-opening and evaluate them both in simulation and on a real robot with little fine-tuning. Compared to open-loop execution, our experiments show that even a limited amount of recovery learning improves task success substantially from 71% to 92.4% in simulation and from 75% to 90% on a real robot.