Grasping as Inference: Reactive Grasping in Heavily Cluttered Environment

TL;DR

This paper presents a reactive, closed-loop grasping framework using a Hidden Markov Model and a lightweight CNN, enabling real-time 6DoF grasp prediction in cluttered environments with improved success rates.

Contribution

It introduces a novel framework combining HMM, particle filtering, and a lightweight CNN for real-time 6DoF grasping in cluttered settings, addressing sequential decision-making.

Findings

Significant improvement in grasp success rate over baseline methods.

Effective real-time inference of 6DoF grasps in cluttered environments.

Reactive system adapts to environmental changes and performs cleanup tasks.

Abstract

Although, in the task of grasping via a data-driven method, closed-loop feedback and predicting 6 degrees of freedom (DoF) grasp rather than conventionally used 4DoF top-down grasp are demonstrated to improve performance individually, few systems have both. Moreover, the sequential property of that task is hardly dealt with, while the approaching motion necessarily generates a series of observations. Therefore, this paper synthesizes three approaches and suggests a closed-loop framework that can predict the 6DoF grasp in a heavily cluttered environment from continuously received vision observations. This can be realized by formulating the grasping problem as Hidden Markov Model and applying a particle filter to infer grasp. Additionally, we introduce a novel lightweight Convolutional Neural Network (CNN) model that evaluates and initializes grasp samples in real-time, making the particle filter process possible. The experiments, which are conducted on a real robot with a heavily cluttered environment, show that our framework not only quantitatively improves the grasping success rate significantly compared to the baseline algorithms, but also qualitatively reacts to a dynamic change in the environment and cleans up the table.