TacSE3: Equivariant SE(3) Motion Estimation from Low-Texture Visuotactile Images for In-Gripper Tracking and Compensation

TL;DR

TacSE3 introduces a novel SE(3) motion estimation method from low-texture visuotactile images, enabling reliable in-gripper object tracking and disturbance compensation in robotic manipulation.

Contribution

It presents a tactile motion-estimation pipeline that decouples translation and rotation from visuotactile data, improving robustness and interpretability for in-hand manipulation.

Findings

Dual-sensor sensing reduces translation-rotation ambiguity.

Supports rotation tracking across axes and geometries.

Enhances disturbance tolerance in manipulation tasks.

Abstract

Robotic in-hand manipulation requires reliable object-motion tracking under frequent visual occlusion, yet low-texture visuotactile images provide few stable correspondences for conventional image- or geometry-matching methods. This paper presents TacSE3, a tactile motion-estimation pipeline that converts low-texture visuotactile observations into a decoupled three-dimensional force field and estimates incremental rigid-body motion on SE(3). The method derives planar translation from contact-centroid motion and estimates rotation primarily from shear-related tactile responses, yielding a physically interpretable signal for in-gripper tracking and compensation. Experiments with paired DM-Tac fingertip sensors show that dual-sensor sensing reduces translation-rotation ambiguity, supports rotation tracking across axes and object geometries, and provides a lightweight compensation signal that improves disturbance tolerance in downstream manipulation tasks without retraining the base policy.