Multi-Step Recurrent Q-Learning for Robotic Velcro Peeling

TL;DR

This paper introduces a multi-step recurrent Q-learning approach enabling robots to perform velcro peeling, a complex non-rigid object manipulation task, by effectively modeling long-term dependencies from noisy sensor data.

Contribution

The work presents a novel deep recurrent learning method for force-based manipulation of non-rigid objects in partially observable environments, validated through real robot experiments.

Findings

Achieves ~90% success rate in velcro peeling tasks.

Outperforms baseline methods significantly.

Demonstrates the importance of modeling long-term dependencies.

Abstract

Learning object manipulation is a critical skill for robots to interact with their environment. Even though there has been significant progress in robotic manipulation of rigid objects, interacting with non-rigid objects remains challenging for robots. In this work, we introduce velcro peeling as a representative application for robotic manipulation of non-rigid objects in complex environments. We present a method of learning force-based manipulation from noisy and incomplete sensor inputs in partially observable environments by modeling long term dependencies between measurements with a multi-step deep recurrent network. We present experiments on a real robot to show the necessity of modeling these long term dependencies and validate our approach in simulation and robot experiments. Our results show that using tactile input enables the robot to overcome geometric uncertainties present in the environment with high fidelity in ~90% of all cases, outperforming the baselines by a large margin.