Haptic-Informed ACT with a Soft Gripper and Recovery-Informed Training for Pseudo Oocyte Manipulation

TL;DR

This paper presents Haptic-Informed ACT, a robotic system that combines haptic feedback, soft grippers, and transformer-based action chunking to improve pseudo oocyte manipulation, achieving higher success and robustness.

Contribution

The study introduces a novel integration of haptic feedback and soft grippers with transformer-based action chunking for enhanced biomedical robotic automation.

Findings

Improved task success rate over traditional methods

Enhanced robustness and adaptability in dynamic environments

Effective real-time grasp failure detection and correction

Abstract

In this paper, we introduce Haptic-Informed ACT, an advanced robotic system for pseudo oocyte manipulation, integrating multimodal information and Action Chunking with Transformers (ACT). Traditional automation methods for oocyte transfer rely heavily on visual perception, often requiring human supervision due to biological variability and environmental disturbances. Haptic-Informed ACT enhances ACT by incorporating haptic feedback, enabling real-time grasp failure detection and adaptive correction. Additionally, we introduce a 3D-printed TPU soft gripper to facilitate delicate manipulations. Experimental results demonstrate that Haptic-Informed ACT improves the task success rate, robustness, and adaptability compared to conventional ACT, particularly in dynamic environments. These findings highlight the potential of multimodal learning in robotics for biomedical automation.