UniVTAC: A Unified Simulation Platform for Visuo-Tactile Manipulation Data Generation, Learning, and Benchmarking

TL;DR

UniVTAC introduces a comprehensive simulation platform and encoder for visuo-tactile data, enabling scalable policy training and benchmarking in robotic manipulation, significantly enhancing success rates in simulated and real-world tasks.

Contribution

The paper presents a novel unified simulation platform, a tactile-centric encoder, and a benchmark for visuo-tactile manipulation, advancing data generation and evaluation in contact-rich robotic tasks.

Findings

Improved success rates by 17.1% on the benchmark.

Real-world experiments show a 25% success rate increase.

Supports scalable and controllable visuo-tactile data synthesis.

Abstract

Robotic manipulation has seen rapid progress with vision-language-action (VLA) policies. However, visuo-tactile perception is critical for contact-rich manipulation, as tasks such as insertion are difficult to complete robustly using vision alone. At the same time, acquiring large-scale and reliable tactile data in the physical world remains costly and challenging, and the lack of a unified evaluation platform further limits policy learning and systematic analysis. To address these challenges, we propose UniVTAC, a simulation-based visuo-tactile data synthesis platform that supports three commonly used visuo-tactile sensors and enables scalable and controllable generation of informative contact interactions. Based on this platform, we introduce the UniVTAC Encoder, a visuo-tactile encoder trained on large-scale simulation-synthesized data with designed supervisory signals, providing tactile-centric visuo-tactile representations for downstream manipulation tasks. In addition, we present the UniVTAC Benchmark, which consists of eight representative visuo-tactile manipulation tasks for evaluating tactile-driven policies. Experimental results show that integrating the UniVTAC Encoder improves average success rates by 17.1% on the UniVTAC Benchmark, while real-world robotic experiments further demonstrate a 25% improvement in task success. Our webpage is available at https://univtac.github.io/.