# Enhancing Robotic Grasping Detection Using Visual–Tactile Fusion Perception

## TL;DR

This paper introduces a new method for robotic grasping by combining visual and tactile data to improve detection accuracy.

## Contribution

A novel visual–tactile fusion method with a Grasp Stability Prediction Module for robotic grasping detection.

## Key findings

- A visual–tactile dataset was constructed with grasp stability for each potential grasping position.
- The proposed method significantly enhances robotic grasping detection accuracy through fusion perception.
- The Grasp Stability Prediction Module provides prior knowledge to improve grasp detection performance.

## Abstract

With the advancement of tactile sensors, researchers increasingly integrate tactile perception into robotics, but only for tasks such as object reconstruction, classification, recognition, and grasp state assessment. In this paper, we rethink the relationship between visual and tactile perception and propose a novel robotic grasping detection method based on visual–tactile perception. Initially, we construct a visual–tactile dataset containing the grasp stability for each potential grasping position. Next, we introduce a novel Grasp Stability Prediction Module (GSPM) to generate a grasp stability probability map, providing prior knowledge regarding grasp stability to the grasp detection network for each possible grasp position. Finally, the map is multiplied element-wise with the corresponding colored image and inputted into the grasp detection network. Experimental results demonstrate that our novel visual–tactile fusion method significantly enhances robotic grasping detection accuracy.