Multi-Head Self-Attending Neural Tucker Factorization

TL;DR

This paper introduces MSNTucF, a neural network-based tensor factorization model that captures complex nonlinear spatiotemporal patterns in dynamic QoS data, significantly improving missing value prediction accuracy.

Contribution

It proposes a novel multi-head self-attending neural Tucker factorization method for modeling intricate nonlinear spatiotemporal interactions in high-dimensional, incomplete tensors.

Findings

Outperforms state-of-the-art models on real QoS datasets

Effectively captures nonlinear spatiotemporal patterns

Demonstrates superior missing value estimation accuracy

Abstract

Quality-of-service (QoS) data exhibit dynamic temporal patterns that are crucial for accurately predicting missing values. These patterns arise from the evolving interactions between users and services, making it essential to capture the temporal dynamics inherent in such data for improved prediction performance. As the size and complexity of QoS datasets increase, existing models struggle to provide accurate predictions, highlighting the need for more flexible and dynamic methods to better capture the underlying patterns in large-scale QoS data. To address this issue, we introduce a neural network-based tensor factorization approach tailored for learning spatiotemporal representations of high-dimensional and incomplete (HDI) tensors, namely the Multi-head Self-attending Neural Tucker Factorization (MSNTucF). The model is elaborately designed for modeling intricate nonlinear spatiotemporal feature interaction patterns hidden in real world data with a two-fold idea. It first employs a neural network structure to generalize the traditional framework of Tucker factorization and then proposes to leverage a multi-head self-attending module to enforce nonlinear latent interaction learning. In empirical studies on two dynamic QoS datasets from real applications, the proposed MSNTucF model demonstrates superior performance compared to state-of-the-art benchmark models in estimating missing observations. This highlights its ability to learn non-linear spatiotemporal representations of HDI tensors.