Predicting Object Interactions with Behavior Primitives: An Application in Stowing Tasks

TL;DR

This paper introduces a novel framework that enables robots to perform complex, long-horizon stowing tasks by learning from a single demonstration, using graph neural networks and primitive-augmented trajectory optimization, with strong simulation-to-real transfer.

Contribution

It presents a generalizable robot stowing policy leveraging object interaction prediction and primitive-based control, reducing data requirements and enhancing adaptability.

Findings

Effective in simulation with few keyframes (3-4) from a single demo.

Generalizes well to real-world variations in shelf and object attributes.

Achieves proficient long-horizon stowing with minimal demonstration data.

Abstract

Stowing, the task of placing objects in cluttered shelves or bins, is a common task in warehouse and manufacturing operations. However, this task is still predominantly carried out by human workers as stowing is challenging to automate due to the complex multi-object interactions and long-horizon nature of the task. Previous works typically involve extensive data collection and costly human labeling of semantic priors across diverse object categories. This paper presents a method to learn a generalizable robot stowing policy from predictive model of object interactions and a single demonstration with behavior primitives. We propose a novel framework that utilizes Graph Neural Networks to predict object interactions within the parameter space of behavioral primitives. We further employ primitive-augmented trajectory optimization to search the parameters of a predefined library of heterogeneous behavioral primitives to instantiate the control action. Our framework enables robots to proficiently execute long-horizon stowing tasks with a few keyframes (3-4) from a single demonstration. Despite being solely trained in a simulation, our framework demonstrates remarkable generalization capabilities. It efficiently adapts to a broad spectrum of real-world conditions, including various shelf widths, fluctuating quantities of objects, and objects with diverse attributes such as sizes and shapes.