Hierarchical and Modular Network on Non-prehensile Manipulation in General Environments

TL;DR

This paper introduces a modular neural network architecture and environment generation method enabling robots to perform non-prehensile manipulation tasks in diverse, unseen environments with zero-shot transfer, advancing generalist robotic manipulation capabilities.

Contribution

It proposes a reconfigurable network architecture and environment generation algorithm to improve generalization of non-prehensile manipulation policies across varied environments.

Findings

Zero-shot transfer to real-world environments achieved

A new benchmark with nine digital twin scenes released

Enhanced policy adaptability to diverse geometric constraints

Abstract

For robots to operate in general environments like households, they must be able to perform non-prehensile manipulation actions such as toppling and rolling to manipulate ungraspable objects. However, prior works on non-prehensile manipulation cannot yet generalize across environments with diverse geometries. The main challenge lies in adapting to varying environmental constraints: within a cabinet, the robot must avoid walls and ceilings; to lift objects to the top of a step, the robot must account for the step's pose and extent. While deep reinforcement learning (RL) has demonstrated impressive success in non-prehensile manipulation, accounting for such variability presents a challenge for the generalist policy, as it must learn diverse strategies for each new combination of constraints. To address this, we propose a modular and reconfigurable architecture that adaptively reconfigures network modules based on task requirements. To capture the geometric variability in environments, we extend the contact-based object representation (CORN) to environment geometries, and propose a procedural algorithm for generating diverse environments to train our agent. Taken together, the resulting policy can zero-shot transfer to novel real-world environments and objects despite training entirely within a simulator. We additionally release a simulation-based benchmark featuring nine digital twins of real-world scenes with 353 objects to facilitate non-prehensile manipulation research in realistic domains.