Self-Supervised Learning of Grasping Arbitrary Objects On-the-Move

TL;DR

This paper presents a self-supervised learning framework enabling mobile robots to grasp arbitrary objects on-the-move by predicting grasp primitives and movement adjustments from visual data, improving accuracy and generalization.

Contribution

It introduces a novel two-stage learning approach with three FCN models for mobile grasping, simplifying complex actions into primitives for better learning and generalization.

Findings

Achieved highest grasping accuracy in experiments.

Enhanced pick-and-place efficiency with the proposed method.

Demonstrated effective generalization through simulation with varied objects.

Abstract

Mobile grasping enhances manipulation efficiency by utilizing robots' mobility. This study aims to enable a commercial off-the-shelf robot for mobile grasping, requiring precise timing and pose adjustments. Self-supervised learning can develop a generalizable policy to adjust the robot's velocity and determine grasp position and orientation based on the target object's shape and pose. Due to mobile grasping's complexity, action primitivization and step-by-step learning are crucial to avoid data sparsity in learning from trial and error. This study simplifies mobile grasping into two grasp action primitives and a moving action primitive, which can be operated with limited degrees of freedom for the manipulator. This study introduces three fully convolutional neural network (FCN) models to predict static grasp primitive, dynamic grasp primitive, and residual moving velocity error from visual inputs. A two-stage grasp learning approach facilitates seamless FCN model learning. The ablation study demonstrated that the proposed method achieved the highest grasping accuracy and pick-and-place efficiency. Furthermore, randomizing object shapes and environments in the simulation effectively achieved generalizable mobile grasping.