Pregrasp Object Material Classification by a Novel Gripper Design with Integrated Spectroscopy

TL;DR

This paper presents a novel robotic gripper with integrated spectroscopy and visual sensing, enabling accurate material classification of objects during grasping with a recursive estimation approach, outperforming visual-only methods.

Contribution

Introduction of a new gripper design with embedded spectral and visual sensors, and a recursive SVM-based classification method for real-time material identification during grasping.

Findings

Achieved 96.4% accuracy in 32-class material classification.

Outperformed state-of-the-art visual algorithms by 29.4%.

Demonstrated effective recursive estimation as the gripper approaches objects.

Abstract

Robots benefit from being able to classify objects they interact with or manipulate based on their material properties. This capability ensures fine manipulation of complex objects through proper grasp pose and force selection. Prior work has focused on haptic or visual processing to determine material type at grasp time. In this work, we introduce a novel parallel robot gripper design and a method for collecting spectral readings and visual images from within the gripper finger. We train a nonlinear Support Vector Machine (SVM) that can classify the material of the object about to be grasped through recursive estimation, with increasing confidence as the distance from the finger tips to the object decreases. In order to validate the hardware design and classification method, we collect samples from 16 real and fake fruit varieties (composed of polystyrene/plastic) resulting in a dataset containing spectral curves, scene images, and high-resolution texture images as the objects are grasped, lifted, and released. Our modeling method demonstrates an accuracy of 96.4% in classifying objects in a 32 class decision problem. This represents a performance improvement by 29.4% over the state of the art computer vision algorithms at distinguishing between visually similar materials. In contrast to prior work, our recursive estimation model accounts for increasing spectral signal strength and allows for decisions to be made as the gripper approaches an object. We conclude that spectroscopy is a promising sensing modality for enabling robots to not only classify grasped objects but also understand their underlying material composition.