Haptic-ACT: Bridging Human Intuition with Compliant Robotic Manipulation via Immersive VR

TL;DR

This paper presents Haptic-ACT, a novel VR-based teleoperation platform for collecting demonstrations and an imitation learning framework that enhances robotic manipulation by incorporating haptic feedback and transformer models.

Contribution

It introduces an immersive VR teleoperation setup for better demonstration collection and a new imitation learning method, Haptic-ACT, that improves robot compliance in manipulation tasks.

Findings

VR platform reduces fingertip forces during demonstrations

Haptic-ACT outperforms original ACT in simulation and real robot tasks

Demonstrations enable more delicate and compliant manipulation

Abstract

Robotic manipulation is essential for the widespread adoption of robots in industrial and home settings and has long been a focus within the robotics community. Advances in artificial intelligence have introduced promising learning-based methods to address this challenge, with imitation learning emerging as particularly effective. However, efficiently acquiring high-quality demonstrations remains a challenge. In this work, we introduce an immersive VR-based teleoperation setup designed to collect demonstrations from a remote human user. We also propose an imitation learning framework called Haptic Action Chunking with Transformers (Haptic-ACT). To evaluate the platform, we conducted a pick-and-place task and collected 50 demonstration episodes. Results indicate that the immersive VR platform significantly reduces demonstrator fingertip forces compared to systems without haptic feedback, enabling more delicate manipulation. Additionally, evaluations of the Haptic-ACT framework in both the MuJoCo simulator and on a real robot demonstrate its effectiveness in teaching robots more compliant manipulation compared to the original ACT. Additional materials are available at https://sites.google.com/view/hapticact.