JENGA: Object selection and pose estimation for robotic grasping from a stack

TL;DR

This paper introduces JENGA, a method for selecting and estimating the pose of objects in stacks for robotic grasping, addressing a less-explored structured environment with a new dataset and evaluation metric.

Contribution

The paper presents a novel camera-IMU based approach for object selection and pose estimation in stacks, along with a benchmarking dataset and evaluation metric.

Findings

Method performs well but is challenging for error-free solutions.

Effective in a brick-picking construction scenario.

Provides a new dataset for structured object stacking environments.

Abstract

Vision-based robotic object grasping is typically investigated in the context of isolated objects or unstructured object sets in bin picking scenarios. However, there are several settings, such as construction or warehouse automation, where a robot needs to interact with a structured object formation such as a stack. In this context, we define the problem of selecting suitable objects for grasping along with estimating an accurate 6DoF pose of these objects. To address this problem, we propose a camera-IMU based approach that prioritizes unobstructed objects on the higher layers of stacks and introduce a dataset for benchmarking and evaluation, along with a suitable evaluation metric that combines object selection with pose accuracy. Experimental results show that although our method can perform quite well, this is a challenging problem if a completely error-free solution is needed. Finally, we show results from the deployment of our method for a brick-picking application in a construction scenario.