Localizing the Object Contact through Matching Tactile Features with Visual Map

TL;DR

This paper introduces a framework that integrates vision and tactile sensing by localizing tactile readings within a visual object map using feature matching and probabilistic estimation, enhancing robotic object interaction capabilities.

Contribution

It proposes a novel method to localize tactile data in visual maps through shared feature descriptors and Bayesian filtering, bridging the gap between tactile and visual perception in robotics.

Findings

Feasibility demonstrated with tactile array sensor tests.

Effective localization of tactile readings in visual maps.

Probabilistic framework improves tactile-visual integration.

Abstract

This paper presents a novel framework for integration of vision and tactile sensing by localizing tactile readings in a visual object map. Intuitively, there are some correspondences, e.g., prominent features, between visual and tactile object identification. To apply it in robotics, we propose to localize tactile readings in visual images by sharing same sets of feature descriptors through two sensing modalities. It is then treated as a probabilistic estimation problem solved in a framework of recursive Bayesian filtering. Feature-based measurement model and Gaussian based motion model are thus built. In our tests, a tactile array sensor is utilized to generate tactile images during interaction with objects and the results have proven the feasibility of our proposed framework.