Foundation Model-Driven Grasping of Unknown Objects via Center of Gravity Estimation

TL;DR

This paper introduces a vision-based grasping approach for unknown objects with uneven mass distribution, using foundation models to accurately localize the center of gravity and improve grasp stability.

Contribution

It develops a CoG localization method leveraging diffusion models and a new dataset, enhancing grasp success rates and generalization over existing techniques.

Findings

49% higher success rate than keypoint-based methods

11% improvement over state-of-the-art affordance methods

76% CoG localization accuracy on unseen objects

Abstract

This study presents a grasping method for objects with uneven mass distribution by leveraging diffusion models to localize the center of gravity (CoG) on unknown objects. In robotic grasping, CoG deviation often leads to postural instability, where existing keypoint-based or affordance-driven methods exhibit limitations. We constructed a dataset of 790 images featuring unevenly distributed objects with keypoint annotations for CoG localization. A vision-driven framework based on foundation models was developed to achieve CoG-aware grasping. Experimental evaluations across real-world scenarios demonstrate that our method achieves a 49\% higher success rate compared to conventional keypoint-based approaches and an 11\% improvement over state-of-the-art affordance-driven methods. The system exhibits strong generalization with a 76\% CoG localization accuracy on unseen objects, providing a novel solution for precise and stable grasping tasks.