FAR-Dex: Few-shot Data Augmentation and Adaptive Residual Policy Refinement for Dexterous Manipulation

TL;DR

FAR-Dex is a hierarchical framework that combines few-shot data augmentation and adaptive policy refinement to improve dexterous manipulation in robotics, achieving higher success rates and better generalization in simulation and real-world tasks.

Contribution

The paper introduces FAR-Dex, a novel hierarchical approach integrating data augmentation and adaptive residual policy refinement for dexterous manipulation.

Findings

Data quality improved by 13.4%

Task success rates increased by 7%

Achieved over 80% success in real-world tasks

Abstract

Achieving human-like dexterous manipulation through the collaboration of multi-fingered hands with robotic arms remains a longstanding challenge in robotics, primarily due to the scarcity of high-quality demonstrations and the complexity of high-dimensional action spaces. To address these challenges, we propose FAR-Dex, a hierarchical framework that integrates few-shot data augmentation with adaptive residual refinement to enable robust and precise arm-hand coordination in dexterous tasks. First, FAR-DexGen leverages the IsaacLab simulator to generate diverse and physically constrained trajectories from a few demonstrations, providing a data foundation for policy training. Second, FAR-DexRes introduces an adaptive residual module that refines policies by combining multi-step trajectory segments with observation features, thereby enhancing accuracy and robustness in manipulation scenarios. Experiments in both simulation and real-world demonstrate that FAR-Dex improves data quality by 13.4% and task success rates by 7% over state-of-the-art methods. It further achieves over 80% success in real-world tasks, enabling fine-grained dexterous manipulation with strong positional generalization.