Robotic Grasping of Fully-Occluded Objects using RF Perception

TL;DR

RF-Grasp is a robotic system that uses RF perception and deep reinforcement learning to grasp fully-occluded objects in unstructured environments, outperforming traditional vision-based methods.

Contribution

It introduces RF perception for occluded object grasping, integrating RF-visual servoing and deep reinforcement learning for complex manipulation tasks.

Findings

Improves success rate and efficiency by 40-50% over baseline.

Enables grasping of fully-occluded objects behind obstacles.

Demonstrates new mechanical search capabilities in unstructured environments.

Abstract

We present the design, implementation, and evaluation of RF-Grasp, a robotic system that can grasp fully-occluded objects in unknown and unstructured environments. Unlike prior systems that are constrained by the line-of-sight perception of vision and infrared sensors, RF-Grasp employs RF (Radio Frequency) perception to identify and locate target objects through occlusions, and perform efficient exploration and complex manipulation tasks in non-line-of-sight settings. RF-Grasp relies on an eye-in-hand camera and batteryless RFID tags attached to objects of interest. It introduces two main innovations: (1) an RF-visual servoing controller that uses the RFID's location to selectively explore the environment and plan an efficient trajectory toward an occluded target, and (2) an RF-visual deep reinforcement learning network that can learn and execute efficient, complex policies for decluttering and grasping. We implemented and evaluated an end-to-end physical prototype of RF-Grasp. We demonstrate it improves success rate and efficiency by up to 40-50% over a state-of-the-art baseline. We also demonstrate RF-Grasp in novel tasks such mechanical search of fully-occluded objects behind obstacles, opening up new possibilities for robotic manipulation. Qualitative results (videos) available at rfgrasp.media.mit.edu