Fusing Visuo-Tactile Perception into Kernelized Synergies for Robust Grasping and Fine Manipulation of Non-rigid Objects

TL;DR

This paper presents an enhanced kernelized synergies framework integrating visuo-tactile perception for improved autonomous grasping and manipulation of non-rigid objects, demonstrating robustness and adaptability in robotic systems.

Contribution

The study introduces a novel integration of visuo-tactile sensing into the kernelized synergies framework, enabling better handling of non-rigid objects in robotic manipulation.

Findings

Successful stable grasping of non-rigid objects

Enhanced manipulation dexterity with tactile feedback

Framework shows good generalization and reusability

Abstract

Handling non-rigid objects using robot hands necessities a framework that does not only incorporate human-level dexterity and cognition but also the multi-sensory information and system dynamics for robust and fine interactions. In this research, our previously developed kernelized synergies framework, inspired from human behaviour on reusing same subspace for grasping and manipulation, is augmented with visuo-tactile perception for autonomous and flexible adaptation to unknown objects. To detect objects and estimate their poses, a simplified visual pipeline using RANSAC algorithm with Euclidean clustering and SVM classifier is exploited. To modulate interaction efforts while grasping and manipulating non-rigid objects, the tactile feedback using T40S shokac chip sensor, generating 3D force information, is incorporated. Moreover, different kernel functions are examined in the kernelized synergies framework, to evaluate its performance and potential against task reproducibility, execution, generalization and synergistic re-usability. Experiments performed with robot arm-hand system validates the capability and usability of upgraded framework on stably grasping and dexterously manipulating the non-rigid objects.