Robustness-Aware Tool Selection and Manipulation Planning with Learned Energy-Informed Guidance

TL;DR

This paper introduces a robustness-aware approach for robotic tool selection and manipulation planning that explicitly optimizes for disturbance resilience using an energy-based robustness metric, validated in simulation and real-world tasks.

Contribution

It presents a hierarchical optimization framework that jointly selects tools and plans trajectories with robustness considerations, a novel approach in tool-use planning.

Findings

Consistently selects robust tools in diverse tasks.

Generates manipulation plans resilient to external disturbances.

Demonstrates effectiveness in both simulation and real-world experiments.

Abstract

Humans subconsciously choose robust ways of selecting and using tools, for example, choosing a ladle over a flat spatula to serve meatballs. However, robustness under external disturbances remains underexplored in robotic tool-use planning. This paper presents a robustness-aware method that jointly selects tools and plans contact-rich manipulation trajectories, explicitly optimizing for robustness against disturbances. At the core of our method is an energy-based robustness metric that guides the planner toward robust manipulation behaviors. We formulate a hierarchical optimization pipeline that first identifies a tool and configuration that optimizes robustness, and then plans a corresponding manipulation trajectory that maintains robustness throughout execution. We evaluate our method across three representative tool-use tasks. Simulation and real-world results demonstrate that our method consistently selects robust tools and generates disturbance-resilient manipulation plans.