Self-supervised Spatio-Temporal Graph Mask-Passing Attention Network for Perceptual Importance Prediction of Multi-point Tactility

TL;DR

This paper introduces a self-supervised spatio-temporal graph neural network model to predict the perceptual importance of tactile points, enhancing haptic information compression for multi-point tactile perception.

Contribution

It presents a novel self-supervised learning approach combined with graph neural networks to model and predict tactile perceptual importance across multiple spatial points.

Findings

Model effectively predicts tactile perceptual importance.

Improves haptic information compression efficiency.

Demonstrates strong performance in multi-point tactile scenarios.

Abstract

While visual and auditory information are prevalent in modern multimedia systems, haptic interaction, e.g., tactile and kinesthetic interaction, provides a unique form of human perception. However, multimedia technology for contact interaction is less mature than non-contact multimedia technologies and requires further development. Specialized haptic media technologies, requiring low latency and bitrates, are essential to enable haptic interaction, necessitating haptic information compression. Existing vibrotactile signal compression methods, based on the perceptual model, do not consider the characteristics of fused tactile perception at multiple spatially distributed interaction points. In fact, differences in tactile perceptual importance are not limited to conventional frequency and time domains, but also encompass differences in the spatial locations on the skin unique to tactile perception. For the most frequently used tactile information, vibrotactile texture perception, we have developed a model to predict its perceptual importance at multiple points, based on self-supervised learning and Spatio-Temporal Graph Neural Network. Current experimental results indicate that this model can effectively predict the perceptual importance of various points in multi-point tactile perception scenarios.