Variational Shape Inference for Grasp Diffusion on SE(3)

TL;DR

This paper introduces a novel framework combining variational shape inference with diffusion models to improve multimodal grasp synthesis in robotics, demonstrating superior performance and robustness in simulation and real-world tasks.

Contribution

It proposes a new approach that leverages variational autoencoders and diffusion models for robust, multimodal grasp synthesis conditioned on object shape.

Findings

Outperforms state-of-the-art methods by 6.3% on ACRONYM dataset.

Demonstrates robustness to point cloud density deterioration.

Achieves 34% more successful grasps in real-world tests.

Abstract

Grasp synthesis is a fundamental task in robotic manipulation which usually has multiple feasible solutions. Multimodal grasp synthesis seeks to generate diverse sets of stable grasps conditioned on object geometry, making the robust learning of geometric features crucial for success. To address this challenge, we propose a framework for learning multimodal grasp distributions that leverages variational shape inference to enhance robustness against shape noise and measurement sparsity. Our approach first trains a variational autoencoder for shape inference using implicit neural representations, and then uses these learned geometric features to guide a diffusion model for grasp synthesis on the SE(3) manifold. Additionally, we introduce a test-time grasp optimization technique that can be integrated as a plugin to further enhance grasping performance. Experimental results demonstrate that our shape inference for grasp synthesis formulation outperforms state-of-the-art multimodal grasp synthesis methods on the ACRONYM dataset by 6.3%, while demonstrating robustness to deterioration in point cloud density compared to other approaches. Furthermore, our trained model achieves zero-shot transfer to real-world manipulation of household objects, generating 34% more successful grasps than baselines despite measurement noise and point cloud calibration errors.