ViTaMIn-B: A Reliable and Efficient Visuo-Tactile Bimanual Manipulation Interface

TL;DR

ViTaMIn-B introduces a robust handheld system with advanced visuo-tactile sensing and pose tracking, enabling efficient data collection and superior performance in complex bimanual manipulation tasks.

Contribution

The paper presents a novel compliant visuo-tactile sensor and a unified pose acquisition method, improving robustness and generalizability for bimanual manipulation data collection.

Findings

High usability confirmed by user studies

Superior task performance in four manipulation tasks

Effective handling of contact-rich interactions

Abstract

Handheld devices have opened up unprecedented opportunities to collect large-scale, high-quality demonstrations efficiently. However, existing systems often lack robust tactile sensing or reliable pose tracking to handle complex interaction scenarios, especially for bimanual and contact-rich tasks. In this work, we propose ViTaMIn-B, a more capable and efficient handheld data collection system for such tasks. We first design DuoTact, a novel compliant visuo-tactile sensor built with a flexible frame to withstand large contact forces during manipulation while capturing high-resolution contact geometry. To enhance the cross-sensor generalizability, we propose reconstructing the sensor's global deformation as a 3D point cloud and using it as the policy input. We further develop a robust, unified 6-DoF bimanual pose acquisition process using Meta Quest controllers, which eliminates the trajectory drift issue in common SLAM-based methods. Comprehensive user studies confirm the efficiency and high usability of ViTaMIn-B among novice and expert operators. Furthermore, experiments on four bimanual manipulation tasks demonstrate its superior task performance relative to existing systems. Project page: https://chuanyune.github.io/ViTaMIn-B_page/