RoboNinja: Learning an Adaptive Cutting Policy for Multi-Material Objects

TL;DR

RoboNinja is a learning-based system that adaptively cuts multi-material objects to maximize yield by removing soft parts while preserving rigid cores, using a closed-loop perception-action approach with a differentiable simulator.

Contribution

The paper introduces RoboNinja, a novel adaptive cutting system with a differentiable simulator for multi-material objects, enabling real-world deployment of learned cutting policies.

Findings

Successfully removes soft material while preserving rigid core.

Adapts cutting actions based on collision feedback and estimated core geometry.

Demonstrates effectiveness on diverse objects with different geometries.

Abstract

We introduce RoboNinja, a learning-based cutting system for multi-material objects (i.e., soft objects with rigid cores such as avocados or mangos). In contrast to prior works using open-loop cutting actions to cut through single-material objects (e.g., slicing a cucumber), RoboNinja aims to remove the soft part of an object while preserving the rigid core, thereby maximizing the yield. To achieve this, our system closes the perception-action loop by utilizing an interactive state estimator and an adaptive cutting policy. The system first employs sparse collision information to iteratively estimate the position and geometry of an object's core and then generates closed-loop cutting actions based on the estimated state and a tolerance value. The "adaptiveness" of the policy is achieved through the tolerance value, which modulates the policy's conservativeness when encountering collisions, maintaining an adaptive safety distance from the estimated core. Learning such cutting skills directly on a real-world robot is challenging. Yet, existing simulators are limited in simulating multi-material objects or computing the energy consumption during the cutting process. To address this issue, we develop a differentiable cutting simulator that supports multi-material coupling and allows for the generation of optimized trajectories as demonstrations for policy learning. Furthermore, by using a low-cost force sensor to capture collision feedback, we were able to successfully deploy the learned model in real-world scenarios, including objects with diverse core geometries and soft materials.