Bag All You Need: Learning a Generalizable Bagging Strategy for Heterogeneous Objects

TL;DR

This paper presents a robotic system with learned policies for heterogeneous bagging, successfully handling rigid and deformable objects, and introduces a simulation benchmark for future research.

Contribution

It introduces a practical robotic solution with two learned policies for complex heterogeneous bagging tasks and provides a new simulation benchmark for the community.

Findings

70% success rate on real-world heterogeneous bagging tasks

Effective learned policies for rearrangement and grasping in complex interactions

A new simulation benchmark for heterogeneous bagging tasks

Abstract

We introduce a practical robotics solution for the task of heterogeneous bagging, requiring the placement of multiple rigid and deformable objects into a deformable bag. This is a difficult task as it features complex interactions between multiple highly deformable objects under limited observability. To tackle these challenges, we propose a robotic system consisting of two learned policies: a rearrangement policy that learns to place multiple rigid objects and fold deformable objects in order to achieve desirable pre-bagging conditions, and a lifting policy to infer suitable grasp points for bi-manual bag lifting. We evaluate these learned policies on a real-world three-arm robot platform that achieves a 70% heterogeneous bagging success rate with novel objects. To facilitate future research and comparison, we also develop a novel heterogeneous bagging simulation benchmark that will be made publicly available.