VBT-MPC: Vision-Based Tactile MPC for Contour Following

TL;DR

This paper introduces VBT-MPC, a novel vision-based tactile control framework for robotic contour following that operates directly on tactile features, eliminating the need for pose estimation or complex force control.

Contribution

The work presents a new tactile control method that uses vision-based tactile sensors and operates in feature space, improving robustness and simplicity over existing approaches.

Findings

VBT-MPC effectively tracks contours on diverse objects in simulation and real-world tests.

Compared to visual-servoing strategies, VBT-MPC shows improved accuracy and robustness.

The approach simplifies tactile control by avoiding pose estimation modules.

Abstract

Tactile sensing plays a key role in robotic manipulation, particularly in tasks like surface inspection. Successful execution requires maintaining contact while accurately tracking object contours. In this work, we propose a Vision-Based Tactile Model Predictive Control (VBT-MPC) framework for robotic contour following using a Vision-Based Tactile Sensor (VBTS) mounted in an eye-in-hand configuration. The proposed controller operates directly in contour features space, thereby avoiding the need for separate pose-estimation modules or complex force-control architectures. We further compare our VBT-MPC with visual-servoing strategies adapted to tactile features, and evaluate contour tracking on objects with diverse geometries and materials in both simulation and real-world experiments.