Embodied Tactile Perception of Soft Objects Properties

TL;DR

This paper investigates how robotic tactile perception of soft objects is influenced by embodiment, multi-modal sensing, and interaction strategies, using a modular e-Skin and an unsupervised deep model to enhance understanding and interpretation.

Contribution

It introduces a modular e-Skin with tunable compliance and multi-modal sensing, and proposes a deep state-space model to interpret interaction dynamics and mechanical properties.

Findings

Multi-modal sensing outperforms uni-modal sensing.

Interaction dynamics significantly influence perception.

The latent filter provides interpretable representations of tactile data.

Abstract

To enable robots to develop human-like fine manipulation, it is essential to understand how mechanical compliance, multi-modal sensing, and purposeful interaction jointly shape tactile perception. In this study, we use a dedicated modular e-Skin with tunable mechanical compliance and multi-modal sensing (normal, shear forces and vibrations) to systematically investigate how sensing embodiment and interaction strategies influence robotic perception of objects. Leveraging a curated set of soft wave objects with controlled viscoelastic and surface properties, we explore a rich set of palpation primitives-pressing, precession, sliding that vary indentation depth, frequency, and directionality. In addition, we propose the latent filter, an unsupervised, action-conditioned deep state-space model of the sophisticated interaction dynamics and infer causal mechanical properties into a structured latent space. This provides generalizable and in-depth interpretable representation of how embodiment and interaction determine and influence perception. Our investigation demonstrates that multi-modal sensing outperforms uni-modal sensing. It highlights a nuanced interaction between the environment and mechanical properties of e-Skin, which should be examined alongside the interaction by incorporating temporal dynamics.