VFAS-Grasp: Closed Loop Grasping with Visual Feedback and Adaptive Sampling

TL;DR

VFAS-Grasp introduces a real-time closed-loop robotic grasping method utilizing visual feedback, adaptive sampling, and uncertainty estimation to enhance grasp success, especially for static and slow-moving objects.

Contribution

The paper presents a novel grasp planning algorithm combining visual feedback, adaptive sampling, and uncertainty-aware scoring for improved real-time robotic grasping.

Findings

Operates at 20 Hz in real time.

Improves grasp success in static scenes.

Enables grasping of slow-moving objects.

Abstract

We consider the problem of closed-loop robotic grasping and present a novel planner which uses Visual Feedback and an uncertainty-aware Adaptive Sampling strategy (VFAS) to close the loop. At each iteration, our method VFAS-Grasp builds a set of candidate grasps by generating random perturbations of a seed grasp. The candidates are then scored using a novel metric which combines a learned grasp-quality estimator, the uncertainty in the estimate and the distance from the seed proposal to promote temporal consistency. Additionally, we present two mechanisms to improve the efficiency of our sampling strategy: We dynamically scale the sampling region size and number of samples in it based on past grasp scores. We also leverage a motion vector field estimator to shift the center of our sampling region. We demonstrate that our algorithm can run in real time (20 Hz) and is capable of improving grasp performance for static scenes by refining the initial grasp proposal. We also show that it can enable grasping of slow moving objects, such as those encountered during human to robot handover.