Can a Machine Feel Vibrations?: A Framework for Vibrotactile Sensation and Emotion Prediction via a Neural Network

TL;DR

This paper introduces a neural network framework that predicts sensory and emotional responses to vibrotactile signals, aiding haptic design by reducing reliance on trial-and-error user testing.

Contribution

It presents a novel dual-stream neural network, VibNet, that accurately predicts user ratings from spectrograms of vibration signals, advancing haptic feedback design.

Findings

VibNet achieved 82% accuracy within user rating deviations.

Spectrogram-based input improved prediction over traditional features.

The framework supports efficient design of vibrotactile icons for specific sensations.

Abstract

Vibrotactile signals offer new possibilities for conveying sensations and emotions in various applications. Yet, designing vibrotactile tactile icons (i.e., Tactons) to evoke specific feelings often requires a trial-and-error process and user studies. To support haptic design, we propose a framework for predicting sensory and emotional ratings from vibration signals. We created 154 Tactons and conducted a study to collect acceleration data from smartphones and roughness, valence, and arousal user ratings (n=36). We converted the Tacton signals into two-channel spectrograms reflecting the spectral sensitivities of mechanoreceptors, then input them into VibNet, our dual-stream neural network. The first stream captures sequential features using recurrent networks, while the second captures temporal-spectral features using 2D convolutional networks. VibNet outperformed baseline models, with 82% of its predictions falling within the standard deviations of ground truth user ratings for two new Tacton sets. We discuss the efficacy of our mechanoreceptive processing and dual-stream neural network and present future research directions.