# Deep Dexterous Grasping of Novel Objects from a Single View

## TL;DR

This paper introduces a comprehensive simulation and evaluation framework for dexterous grasping of novel objects from a single view, demonstrating high success rates both in simulation and on real robots.

## Contribution

It provides a new simulator, a large dataset of simulated grasps, multiple neural architectures for grasp generation and evaluation, and real robot validation of the best models.

## Key findings

- Achieved up to 90.49% grasp success rate in simulation.
- Real robot experiments reached 87.8% success rate.
- Significant improvement over baseline methods.

## Abstract

Dexterous grasping of a novel object given a single view is an open problem. This paper makes several contributions to its solution. First, we present a simulator for generating and testing dexterous grasps. Second we present a data set, generated by this simulator, of 2.4 million simulated dexterous grasps of variations of 294 base objects drawn from 20 categories. Third, we present a basic architecture for generation and evaluation of dexterous grasps that may be trained in a supervised manner. Fourth, we present three different evaluative architectures, employing ResNet-50 or VGG16 as their visual backbone. Fifth, we train, and evaluate seventeen variants of generative-evaluative architectures on this simulated data set, showing improvement from 69.53% grasp success rate to 90.49%. Finally, we present a real robot implementation and evaluate the four most promising variants, executing 196 real robot grasps in total. We show that our best architectural variant achieves a grasp success rate of 87.8% on real novel objects seen from a single view, improving on a baseline of 57.1%.