Sim-to-Real Gentle Manipulation of Deformable and Fragile Objects with Stress-Guided Reinforcement Learning

TL;DR

This paper introduces a stress-aware reinforcement learning approach for gentle, damage-free manipulation of deformable and fragile objects, successfully transferring policies from simulation to real-world tasks without additional training.

Contribution

It proposes a novel stress-penalized reward and curriculum learning strategy for sim-to-real transfer in delicate object manipulation tasks.

Findings

Reduced applied stress by 36.5% compared to standard RL

Zero-shot transfer from simulation to real-world tasks

Effective manipulation of fragile objects like tofu

Abstract

Robotic manipulation of deformable and fragile objects presents significant challenges, as excessive stress can lead to irreversible damage to the object. While existing solutions rely on accurate object models or specialized sensors and grippers, this adds complexity and often lacks generalization. To address this problem, we present a vision-based reinforcement learning approach that incorporates a stress-penalized reward to discourage damage to the object explicitly. In addition, to bootstrap learning, we incorporate offline demonstrations as well as a designed curriculum progressing from rigid proxies to deformables. We evaluate the proposed method in both simulated and real-world scenarios, showing that the policy learned in simulation can be transferred to the real world in a zero-shot manner, performing tasks such as picking up and pushing tofu. Our results show that the learned policies exhibit a damage-aware, gentle manipulation behavior, demonstrating their effectiveness by decreasing the stress applied to fragile objects by 36.5% while achieving the task goals, compared to vanilla RL policies.