A versatile robotic hand with 3D perception, force sensing for autonomous manipulation

TL;DR

This paper presents a versatile, open-source robotic hand with integrated 3D perception and force sensing, enabling autonomous manipulation tasks like assembly and stacking with robust control and re-planning capabilities.

Contribution

The paper introduces a novel robotic hand with integrated perception and force sensing, along with a complete autonomous manipulation pipeline emphasizing versatility and open-source design.

Findings

Force control up to 32N with 0.08N precision

Successful demonstration of assembly and stacking tasks

Robust long-sequence sensor-based manipulation achieved

Abstract

We describe a force-controlled robotic gripper with built-in tactile and 3D perception. We also describe a complete autonomous manipulation pipeline consisting of object detection, segmentation, point cloud processing, force-controlled manipulation, and symbolic (re)-planning. The design emphasizes versatility in terms of applications, manufacturability, use of commercial off-the-shelf parts, and open-source software. We validate the design by characterizing force control (achieving up to 32N, controllable in steps of 0.08N), force measurement, and two manipulation demonstrations: assembly of the Siemens gear assembly problem, and a sensor-based stacking task requiring replanning. These demonstrate robust execution of long sequences of sensor-based manipulation tasks, which makes the resulting platform a solid foundation for researchers in task-and-motion planning, educators, and quick prototyping of household, industrial and warehouse automation tasks.