Data: WireFishing-M: A multimodal dataset for deformable cable insertion using tactile, visual, and proprioceptive sensing

TL;DR

WireFishing-M is a new dataset for robotic tasks involving deformable objects, combining tactile, visual, and proprioceptive data during cable insertion.

Contribution

The novelty lies in providing a synchronized multimodal dataset for deformable object manipulation with tactile, visual, and proprioceptive sensing.

Findings

The dataset includes seven cable types with synchronized tactile, visual, and joint state data.

WireFishing-M supports multimodal model development for perception and manipulation tasks.

The dataset is publicly available for benchmarking and learning in robotic applications.

Abstract

This article introduces WireFishing-M, a multimodal dataset designed to support research in deformable object manipulation, tactile sensing, and contact-rich robotic tasks. The dataset captures a robotic wire insertion scenario, where a Franka Emika Panda 7-DOF robotic manipulator equipped with an Allegro Robot Hand and a DIGIT tactile sensor performs repeated cable insertions into a transparent l-shaped PVC pipe (1″ Sch 40 NSF-61). The dataset includes seven different types of cables varying in physical properties. For each cable, we collected synchronized GelSight tactile images, multi-view RGB videos (front view, bottom view, and a side view monitoring the pipe opening for insertion outcome), end-effector poses, robot joint states, and externally estimated forces at the end-effector. The tactile sensor continuously captures contact interactions as the robot grips, inserts, and exits the cable from the pipe. WireFishing-M enables the development and benchmarking of multimodal models for perception, force estimation, manipulation policy learning, and success detection in deformable object tasks. The complete dataset is publicly available via Harvard Dataverse and is organized to facilitate direct use in robotic learning and simulation frameworks.

Figures