CAGE: Context-Aware Grasping Engine

TL;DR

This paper introduces a novel context-aware grasping engine that uses a semantic representation and a neural network to improve robotic grasping based on object and task contexts, validated on a large dataset and real robot experiments.

Contribution

The paper presents a new semantic representation and neural network model for context-aware grasping, outperforming prior methods on a large dataset and real robot tasks.

Findings

Outperformed all baseline methods with significant margins.

Successfully achieved 31 of 32 suitable grasps in robot experiments.

Validated effectiveness on a dataset of 14,000 grasps across various objects and tasks.

Abstract

Semantic grasping is the problem of selecting stable grasps that are functionally suitable for specific object manipulation tasks. In order for robots to effectively perform object manipulation, a broad sense of contexts, including object and task constraints, needs to be accounted for. We introduce the Context-Aware Grasping Engine, which combines a novel semantic representation of grasp contexts with a neural network structure based on the Wide & Deep model, capable of capturing complex reasoning patterns. We quantitatively validate our approach against three prior methods on a novel dataset consisting of 14,000 semantic grasps for 44 objects, 7 tasks, and 6 different object states. Our approach outperformed all baselines by statistically significant margins, producing new insights into the importance of balancing memorization and generalization of contexts for semantic grasping. We further demonstrate the effectiveness of our approach on robot experiments in which the presented model successfully achieved 31 of 32 suitable grasps. The code and data are available at: https://github.com/wliu88/rail_semantic_grasping