SmartHand: Towards Embedded Smart Hands for Prosthetic and Robotic Applications

TL;DR

SmartHand introduces an embedded tactile sensing system with high-resolution data processing for prosthetic and robotic hands, achieving high accuracy with low latency and power consumption.

Contribution

The paper presents a novel embedded system with a compact neural network for real-time tactile data classification, improving resolution and efficiency over existing solutions.

Findings

Achieved up to 98.86% top-1 accuracy in tactile classification.

Reduced computational requirements by 15.6x compared to related work.

Maintained high accuracy with low power consumption (505mW).

Abstract

The sophisticated sense of touch of the human hand significantly contributes to our ability to safely, efficiently, and dexterously manipulate arbitrary objects in our environment. Robotic and prosthetic devices lack refined, tactile feedback from their end-effectors, leading to counterintuitive and complex control strategies. To address this lack, tactile sensors have been designed and developed, but they often offer an insufficient spatial and temporal resolution. This paper focuses on overcoming these issues by designing a smart embedded system, called SmartHand, enabling the acquisition and real-time processing of high-resolution tactile information from a hand-shaped multi-sensor array for prosthetic and robotic applications. We acquire a new tactile dataset consisting of 340,000 frames while interacting with 16 everyday objects and the empty hand, i.e., a total of 17 classes. The design of the embedded system minimizes response latency in classification, by deploying a small yet accurate convolutional neural network on a high-performance ARM Cortex-M7 microcontroller. Compared to related work, our model requires one order of magnitude less memory and 15.6x fewer computations while achieving similar inter-session accuracy and up to 98.86% and 99.83% top-1 and top-3 cross-validation accuracy, respectively. Experimental results show a total power consumption of 505mW and a latency of only 100ms.