Vision and Control for Grasping Clear Plastic Bags

TL;DR

This paper presents two novel vision techniques, one classical and one deep learning-based, for planning grasps on clear plastic bags, combined with a control method for executing these grasps with a Sawyer robot arm.

Contribution

It introduces a deep learning approach for grasp planning on transparent objects and integrates it with a control method for robotic execution.

Findings

Deep learning method outperforms classical image processing.

Clustering improves grasp target accuracy.

Successful grasp execution with Sawyer arm.

Abstract

We develop two novel vision methods for planning effective grasps for clear plastic bags, as well as a control method to enable a Sawyer arm with a parallel gripper to execute the grasps. The first vision method is based on classical image processing and heuristics (e.g., Canny edge detection) to select a grasp target and angle. The second uses a deep-learning model trained on a human-labeled data set to mimic human grasp decisions. A clustering algorithm is used to de-noise the outputs of each vision method. Subsequently, a workspace PD control method is used to execute each grasp. Of the two vision methods, we find the deep-learning based method to be more effective.