Estimating Fingertip Forces, Torques, and Local Curvatures from Fingernail Images

TL;DR

This paper presents a novel image-based method to estimate fingertip forces, torques, and surface curvatures from fingernail images, enabling contact analysis without specialized sensors or gloves.

Contribution

It introduces a new approach using fingernail images to estimate contact parameters, extending previous single-finger methods to multiple fingers for grasping analysis.

Findings

High accuracy in force, torque, and surface curvature prediction

Effective multi-finger force estimation for grasping analysis

Method circumvents limitations of sensorized objects and gloves

Abstract

The study of dexterous manipulation has provided important insights in humans sensorimotor control as well as inspiration for manipulation strategies in robotic hands. Previous work focused on experimental environment with restrictions. Here we describe a method using the deformation and color distribution of the fingernail and its surrounding skin, to estimate the fingertip forces, torques and contact surface curvatures for various objects, including the shape and material of the contact surfaces and the weight of the objects. The proposed method circumvents limitations associated with sensorized objects, gloves or fixed contact surface type. In addition, compared with previous single finger estimation in an experimental environment, we extend the approach to multiple finger force estimation, which can be used for applications such as human grasping analysis. Four algorithms are used, c.q., Gaussian process (GP), Convolutional Neural Networks (CNN), Neural Networks with Fast Dropout (NN-FD) and Recurrent Neural Networks with Fast Dropout (RNN-FD), to model a mapping from images to the corresponding labels. The results further show that the proposed method has high accuracy to predict force, torque and contact surface.