DEFT: Dexterous Fine-Tuning for Real-World Hand Policies

TL;DR

DEFT introduces a novel method combining human priors and online optimization to enable data-efficient, real-world dexterous manipulation with soft robotic hands across complex tasks.

Contribution

The paper presents DEFT, a new approach that leverages human-driven priors and online fine-tuning for robust, data-efficient dexterous manipulation in real-world scenarios.

Findings

Successful manipulation of soft, deformable objects.

Demonstrated robustness across various complex tasks.

Achieved data efficiency through combined priors and online learning.

Abstract

Dexterity is often seen as a cornerstone of complex manipulation. Humans are able to perform a host of skills with their hands, from making food to operating tools. In this paper, we investigate these challenges, especially in the case of soft, deformable objects as well as complex, relatively long-horizon tasks. However, learning such behaviors from scratch can be data inefficient. To circumvent this, we propose a novel approach, DEFT (DExterous Fine-Tuning for Hand Policies), that leverages human-driven priors, which are executed directly in the real world. In order to improve upon these priors, DEFT involves an efficient online optimization procedure. With the integration of human-based learning and online fine-tuning, coupled with a soft robotic hand, DEFT demonstrates success across various tasks, establishing a robust, data-efficient pathway toward general dexterous manipulation. Please see our website at https://dexterous-finetuning.github.io for video results.