Learning 6-DoF Grasping and Pick-Place Using Attention Focus

TL;DR

This paper presents a novel approach for 6-DoF robotic grasping and pick-and-place tasks using attention mechanisms, formulated as an MDP with hierarchical sampling to focus on task-relevant scene parts, trained in simulation and tested on real robots.

Contribution

It introduces hierarchical SE(3) sampling (HSE3S) for attention-focused learning in 6-DoF manipulation tasks, enabling effective transfer from simulation to real-world scenarios.

Findings

Successful in simulation and real robot experiments

Effective handling of cluttered scenes with novel objects

Improved focus on task-relevant scene parts

Abstract

We address a class of manipulation problems where the robot perceives the scene with a depth sensor and can move its end effector in a space with six degrees of freedom -- 3D position and orientation. Our approach is to formulate the problem as a Markov decision process (MDP) with abstract yet generally applicable state and action representations. Finding a good solution to the MDP requires adding constraints on the allowed actions. We develop a specific set of constraints called hierarchical $\text{SE}(3)$ sampling (HSE3S) which causes the robot to learn a sequence of gazes to focus attention on the task-relevant parts of the scene. We demonstrate the effectiveness of our approach on three challenging pick-place tasks (with novel objects in clutter and nontrivial places) both in simulation and on a real robot, even though all training is done in simulation.