Scalable Dexterous Robot Learning with AR-based Remote Human-Robot Interactions

TL;DR

This paper introduces a scalable framework for dexterous robot manipulation that combines AR-based remote human demonstrations with contrastive reinforcement learning, resulting in faster, safer, and more effective policies validated in simulation and real-world tests.

Contribution

It presents a novel two-phase learning approach integrating behavior cloning from AR demonstrations and contrastive RL, enhancing efficiency and robustness in dexterous robot manipulation.

Findings

Significantly faster inference compared to classic methods

Achieves higher success rates in manipulation tasks

Overcomes policy collapse with contrastive learning

Abstract

This paper focuses on the scalable robot learning for manipulation in the dexterous robot arm-hand systems, where the remote human-robot interactions via augmented reality (AR) are established to collect the expert demonstration data for improving efficiency. In such a system, we present a unified framework to address the general manipulation task problem. Specifically, the proposed method consists of two phases: i) In the first phase for pretraining, the policy is created in a behavior cloning (BC) manner, through leveraging the learning data from our AR-based remote human-robot interaction system; ii) In the second phase, a contrastive learning empowered reinforcement learning (RL) method is developed to obtain more efficient and robust policy than the BC, and thus a projection head is designed to accelerate the learning progress. An event-driven augmented reward is adopted for enhancing the safety. To validate the proposed method, both the physics simulations via PyBullet and real-world experiments are carried out. The results demonstrate that compared to the classic proximal policy optimization and soft actor-critic policies, our method not only significantly speeds up the inference, but also achieves much better performance in terms of the success rate for fulfilling the manipulation tasks. By conducting the ablation study, it is confirmed that the proposed RL with contrastive learning overcomes policy collapse. Supplementary demonstrations are available at https://cyberyyc.github.io/.