A Soft Humanoid Hand with In-Finger Visual Perception

TL;DR

This paper introduces a soft humanoid hand with integrated fingertip cameras and onboard visual processing, demonstrating durable grasping and high-accuracy object segmentation for advanced robotic manipulation.

Contribution

The paper presents a novel soft humanoid hand with embedded visual sensors and a hybrid FPGA-microcontroller system for real-time image processing, advancing soft robotics capabilities.

Findings

Grasp force of 31.8 N achieved

Finger durability exceeds 15,000 cycles

Object segmentation accuracy above 90%

Abstract

We present a novel underactued humanoid five finger soft hand, the KIT \softhand, which is equipped with cameras in the fingertips and integrates a high performance embedded system for visual processing and control. We describe the actuation mechanism of the hand and the tendon-driven soft finger design with internally routed high-bandwidth flat-flex cables. For efficient on-board parallel processing of visual data from the cameras in each fingertip, we present a hybrid embedded architecture consisting of a field programmable logic array (FPGA) and a microcontroller that allows the realization of visual object segmentation based on convolutional neural networks. We evaluate the hand design by conducting durability experiments with one finger and quantify the grasp performance in terms of grasping force, speed and grasp success. The results show that the hand exhibits a grasp force of 31.8 N and a mechanical durability of the finger of more than 15.000 closing cycles. Finally, we evaluate the accuracy of visual object segmentation during the different phases of the grasping process using five different objects. Hereby, an accuracy above 90 % can be achieved.