Gentle Object Retraction in Dense Clutter Using Multimodal Force Sensing and Imitation Learning

TL;DR

This paper explores how multimodal force sensing combined with imitation learning enables robots to gently and effectively retract objects from dense clutter, improving success rates and safety compared to vision-only approaches.

Contribution

It introduces a multimodal force sensing approach integrated with imitation learning for safe object retraction in cluttered environments, demonstrating significant performance improvements.

Findings

Force sensing reduces excessive force failures.

Combined tactile and wrench information improves success rate by 80%.

Force-enabled policies are faster and more reliable.

Abstract

Dense collections of movable objects are common in everyday spaces-from cabinets in a home to shelves in a warehouse. Safely retracting objects from such collections is difficult for robots, yet people do it frequently, leveraging learned experience in tandem with vision and non-prehensile tactile sensing on the sides and backs of their hands and arms. We investigate the role of contact force sensing for training robots to gently reach into constrained clutter and extract objects. The available sensing modalities are (1) "eye-in-hand" vision, (2) proprioception, (3) non-prehensile triaxial tactile sensing, (4) contact wrenches estimated from joint torques, and (5) a measure of object acquisition obtained by monitoring the vacuum line of a suction cup. We use imitation learning to train policies from a set of demonstrations on randomly generated scenes, then conduct an ablation study of wrench and tactile information. We evaluate each policy's performance across 40 unseen environment configurations. Policies employing any force sensing show fewer excessive force failures, an increased overall success rate, and faster completion times. The best performance is achieved using both tactile and wrench information, producing an 80% improvement above the baseline without force information.