Symmetry-Aware Fusion of Vision and Tactile Sensing via Bilateral Force Priors for Robotic Manipulation

TL;DR

This paper introduces a symmetry-aware visuo-tactile fusion method using a Cross-Modal Transformer with physics-informed regularization, significantly improving robotic insertion success rates by effectively combining vision and tactile feedback.

Contribution

It proposes a novel CMT architecture with bilateral force regularization for stable visuo-tactile fusion in robotic manipulation tasks.

Findings

Achieves 96.59% insertion success rate on TacSL benchmark.

Outperforms naive and gated fusion baselines.

Approaches privileged sensor configuration performance.

Abstract

Insertion tasks in robotic manipulation demand precise, contact-rich interactions that vision alone cannot resolve. While tactile feedback is intuitively valuable, existing studies have shown that na\"ive visuo-tactile fusion often fails to deliver consistent improvements. In this work, we propose a Cross-Modal Transformer (CMT) for visuo-tactile fusion that integrates wrist-camera observations with tactile signals through structured self- and cross-attention. To stabilize tactile embeddings, we further introduce a physics-informed regularization that encourages bilateral force balance, reflecting principles of human motor control. Experiments on the TacSL benchmark show that CMT with symmetry regularization achieves a 96.59% insertion success rate, surpassing na\"ive and gated fusion baselines and closely matching the privileged "wrist + contact force" configuration (96.09%). These results highlight two central insights: (i) tactile sensing is indispensable for precise alignment, and (ii) principled multimodal fusion, further strengthened by physics-informed regularization, unlocks complementary strengths of vision and touch, approaching privileged performance under realistic sensing.