A Learning-based Controller for Multi-Contact Grasps on Unknown Objects with a Dexterous Hand

TL;DR

This paper introduces a learning-based multi-contact grasp controller for dexterous hands that operates on unknown objects using minimal prior information, achieving high stability and real-time performance.

Contribution

It presents a novel controller supporting arbitrary grasps on unseen objects, utilizing neural networks for wrench estimation and torque prediction based on minimal object data.

Findings

Achieves 83.1% grasp stability under 10N external forces.

Outperforms baseline methods in efficiency and stability.

Operates at 6ms cycle time on real hardware.

Abstract

Existing grasp controllers usually either only support finger-tip grasps or need explicit configuration of the inner forces. We propose a novel grasp controller that supports arbitrary grasp types, including power grasps with multi-contacts, while operating self-contained on before unseen objects. No detailed contact information is needed, but only a rough 3D model, e.g., reconstructed from a single depth image. First, the external wrench being applied to the object is estimated by using the measured torques at the joints. Then, the torques necessary to counteract the estimated wrench while keeping the object at its initial pose are predicted. The torques are commanded via desired joint angles to an underlying joint-level impedance controller. To reach real-time performance, we propose a learning-based approach that is based on a wrench estimator- and a torque predictor neural network. Both networks are trained in a supervised fashion using data generated via the analytical formulation of the controller. In an extensive simulation-based evaluation, we show that our controller is able to keep 83.1% of the tested grasps stable when applying external wrenches with up to 10N. At the same time, we outperform the two tested baselines by being more efficient and inducing less involuntary object movement. Finally, we show that the controller also works on the real DLR-Hand II, reaching a cycle time of 6ms.