Object-Focus Actor for Data-efficient Robot Generalization Dexterous Manipulation

TL;DR

This paper introduces Object-Focus Actor (OFA), a data-efficient method for dexterous robot manipulation that generalizes well across diverse scenes and object placements using minimal demonstrations.

Contribution

OFA leverages consistent end trajectories in manipulation tasks and employs a hierarchical pipeline for efficient policy training and execution, improving generalization and data efficiency.

Findings

Outperforms baseline methods in real-world tests

Achieves robust performance with only 10 demonstrations

Demonstrates significant generalization across varied backgrounds and object positions

Abstract

Robot manipulation learning from human demonstrations offers a rapid means to acquire skills but often lacks generalization across diverse scenes and object placements. This limitation hinders real-world applications, particularly in complex tasks requiring dexterous manipulation. Vision-Language-Action (VLA) paradigm leverages large-scale data to enhance generalization. However, due to data scarcity, VLA's performance remains limited. In this work, we introduce Object-Focus Actor (OFA), a novel, data-efficient approach for generalized dexterous manipulation. OFA exploits the consistent end trajectories observed in dexterous manipulation tasks, allowing for efficient policy training. Our method employs a hierarchical pipeline: object perception and pose estimation, pre-manipulation pose arrival and OFA policy execution. This process ensures that the manipulation is focused and efficient, even in varied backgrounds and positional layout. Comprehensive real-world experiments across seven tasks demonstrate that OFA significantly outperforms baseline methods in both positional and background generalization tests. Notably, OFA achieves robust performance with only 10 demonstrations, highlighting its data efficiency.