Learning 6-DoF Grasping and Pick-Place Using Attention Focus
Marcus Gualtieri, Robert Platt

TL;DR
This paper presents a novel approach for 6-DoF robotic grasping and pick-and-place tasks using attention mechanisms, formulated as an MDP with hierarchical sampling to focus on task-relevant scene parts, trained in simulation and tested on real robots.
Contribution
It introduces hierarchical SE(3) sampling (HSE3S) for attention-focused learning in 6-DoF manipulation tasks, enabling effective transfer from simulation to real-world scenarios.
Findings
Successful in simulation and real robot experiments
Effective handling of cluttered scenes with novel objects
Improved focus on task-relevant scene parts
Abstract
We address a class of manipulation problems where the robot perceives the scene with a depth sensor and can move its end effector in a space with six degrees of freedom -- 3D position and orientation. Our approach is to formulate the problem as a Markov decision process (MDP) with abstract yet generally applicable state and action representations. Finding a good solution to the MDP requires adding constraints on the allowed actions. We develop a specific set of constraints called hierarchical sampling (HSE3S) which causes the robot to learn a sequence of gazes to focus attention on the task-relevant parts of the scene. We demonstrate the effectiveness of our approach on three challenging pick-place tasks (with novel objects in clutter and nontrivial places) both in simulation and on a real robot, even though all training is done in simulation.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsRobot Manipulation and Learning · Reinforcement Learning in Robotics · Domain Adaptation and Few-Shot Learning
