# Human-taught sensory-control synergy for universal robotic grasping

## TL;DR

This paper introduces a human-like system for robots to learn grasping skills from humans, enabling them to handle a wide variety of objects effectively.

## Contribution

The novel sensory-control synergy (SCS) approach encodes human tactile data into semantic states for adaptive robotic grasping.

## Key findings

- The robot achieved a 95.2% success rate in grasping diverse objects.
- The SCS model is data-efficient and transferable to robots for universal manipulation.
- The system emulates human neurocognition and motor control strategies.

## Abstract

Universal grasping is essential but challenging in robotic manipulations, particularly for humanoid robots with multifingered hands. To learn skills of dexterous manipulations from humans, we propose a sensory-control synergy (SCS) approach mimicking the human grasping experience. We develop a tactile glove worn on a human hand to capture multimodal tactile data (contact, slip and pressure) during human grasping demonstrations. Emulating human neurocognition, the multimodal tactile data are encoded into semantically explicit grasping states by neural-network computing. Drawing on human motor control strategies, an experience-based fuzzy controller is built to swiftly convert semantic grasping states into grasping actions. Benefiting from the semantization of grasping states, the SCS model is highly logicalized and generalizable, can be data-efficiently built by non-experts and readily transferred to robots for accomplishing universal robotic manipulation. The robot with SCS achieves an average success rate of 95.2% in grasping diverse objects of daily life, including slippery, fragile, soft and heavy objects. In dynamic disturbance and complex tasks, the robot autonomously manipulates using its adaptive SCS, demonstrating human-like universal grasping capabilities.

Human-like sensory-control synergy (SCS) approach enables robots to data-efficiently learn skills of dexterous manipulations from humans, achieving adaptive and universal grasping diverse objects of daily life.