VMGNet: A Low Computational Complexity Robotic Grasping Network Based on VMamba with Multi-Scale Feature Fusion

TL;DR

VMGNet is a novel low-complexity, high-accuracy robotic grasping network that introduces the Visual State Space for linear computational complexity and employs multi-scale feature fusion for improved performance.

Contribution

The paper presents VMGNet, the first model to incorporate Visual State Space for linear complexity and a lightweight Fusion Bridge Module for multi-scale feature fusion in robotic grasping.

Findings

VMGNet achieves only 8.7G FLOPs and 8.1 ms inference time.

It attains state-of-the-art results on Cornell and Jacquard datasets.

Real-world grasping success rate of 94.4%.

Abstract

While deep learning-based robotic grasping technology has demonstrated strong adaptability, its computational complexity has also significantly increased, making it unsuitable for scenarios with high real-time requirements. Therefore, we propose a low computational complexity and high accuracy model named VMGNet for robotic grasping. For the first time, we introduce the Visual State Space into the robotic grasping field to achieve linear computational complexity, thereby greatly reducing the model's computational cost. Meanwhile, to improve the accuracy of the model, we propose an efficient and lightweight multi-scale feature fusion module, named Fusion Bridge Module, to extract and fuse information at different scales. We also present a new loss function calculation method to enhance the importance differences between subtasks, improving the model's fitting ability. Experiments show that VMGNet has only 8.7G Floating Point Operations and an inference time of 8.1 ms on our devices. VMGNet also achieved state-of-the-art performance on the Cornell and Jacquard public datasets. To validate VMGNet's effectiveness in practical applications, we conducted real grasping experiments in multi-object scenarios, and VMGNet achieved an excellent performance with a 94.4% success rate in real-world grasping tasks. The video for the real-world robotic grasping experiments is available at https://youtu.be/S-QHBtbmLc4.