Elly: A Real-Time Failure Recovery and Data Collection System for Robotic Manipulation

TL;DR

Elly is a system that enables real-time failure recovery and data collection in robotic manipulation tasks through haptic human intervention, improving robustness and learning from failures.

Contribution

This work introduces Elly, a novel system integrating haptic intervention for failure recovery and data collection during autonomous robot operations.

Findings

Achieved over 80% task completion in experiments

Validated on single-arm and bimanual tasks

Collected data for failure analysis and prevention

Abstract

Even the most robust autonomous behaviors can fail. The goal of this research is to both recover and collect data from failures, during autonomous task execution, so they can be prevented in the future. We propose haptic intervention for real-time failure recovery and data collection. Elly is a system that allows for seamless transitions between autonomous robot behaviors and human intervention while collecting sensory information from the human's recovery strategy. The system and our design choices were experimentally validated on a single arm task -- installing a lightbulb in a socket -- and a bimanual task -- screwing a cap on a bottle -- using two 7-DOF manipulators equipped 4-finger grippers. In these examples, Elly achieved over 80% task completion during a total of 40 runs.