xHAP: Cross-Modal Attention for Haptic Feedback Estimation in the Tactile Internet

TL;DR

This paper introduces xHAP, a deep learning model using cross-modal attention to accurately estimate haptic feedback in the Tactile Internet, improving reliability and coverage under challenging network conditions.

Contribution

The paper presents a novel cross-modal attention architecture for haptic feedback estimation, significantly reducing error and enhancing network robustness in tactile internet applications.

Findings

Reduces mean-squared error by over two orders of magnitude.

Lowers SNR requirement by 10 dB for reliable transmission.

Increases coverage by 138% and supports 59.6% more users.

Abstract

The Tactile Internet requires ultra-low latency and high-fidelity haptic feedback to enable immersive teleoperation. A key challenge is to ensure ultra-reliable and low-latency transmission of haptic packets under channel variations and potential network outages. To address these issues, one approach relies on local estimation of haptic feedback at the operator side. However, designing an accurate estimator that can faithfully reproduce the true haptic forces remains a significant challenge. In this paper, we propose a novel deep learning architecture, xHAP, based on cross-modal attention to estimate haptic feedback. xHAP fuses information from two distinct data streams: the teleoperator's historical force feedback and the operator's control action sequence. We employ modality-specific encoders to learn temporal representations, followed by a cross-attention layer where the teleoperator haptic data attend to the operator input. This fusion allows the model to selectively focus on the most relevant operator sensory data when predicting the teleoperator's haptic feedback. The proposed architecture reduces the mean-squared error by more than two orders of magnitude compared to existing methods and lowers the SNR requirement for reliable transmission by $10~\mathrm{dB}$ at an error threshold of $0.1$ in a 3GPP UMa scenario. Additionally, it increases coverage by $138\%$ and supports $59.6\%$ more haptic users even under 10 dB lower SNR compared to the baseline.