Large Language Model Meets Graph Neural Network in Knowledge Distillation

TL;DR

This paper introduces TOGCL, a novel framework combining graph neural networks and Transformers to improve temporal QoS prediction by modeling high-order relationships and dynamic feature learning, significantly outperforming existing methods.

Contribution

The paper proposes a dynamic graph-based framework with a target-prompt graph attention network and Transformer encoder for enhanced temporal QoS prediction, addressing previous limitations.

Findings

Achieves up to 38.80% performance improvement over state-of-the-art methods.

Effectively models high-order relationships and temporal feature evolution.

Demonstrates robustness and accuracy on WS-DREAM dataset.

Abstract

In service-oriented architectures, accurately predicting the Quality of Service (QoS) is crucial for maintaining reliability and enhancing user satisfaction. However, significant challenges remain due to existing methods always overlooking high-order latent collaborative relationships between users and services and failing to dynamically adjust feature learning for every specific user-service invocation, which are critical for learning accurate features. Additionally, reliance on RNNs for capturing QoS evolution hampers models' ability to detect long-term trends due to difficulties in managing long-range dependencies. To address these challenges, we propose the \underline{T}arget-Prompt \underline{O}nline \underline{G}raph \underline{C}ollaborative \underline{L}earning (TOGCL) framework for temporal-aware QoS prediction. TOGCL leverages a dynamic user-service invocation graph to model historical interactions, providing a comprehensive representation of user-service relationships. Building on this graph, it develops a target-prompt graph attention network to extract online deep latent features of users and services at each time slice, simultaneously considering implicit collaborative relationships between target users/services and their neighbors, as well as relevant historical QoS values. Additionally, a multi-layer Transformer encoder is employed to uncover temporal feature evolution patterns of users and services, leading to temporal-aware QoS prediction. Extensive experiments conducted on the WS-DREAM dataset demonstrate that our proposed TOGCL framework significantly outperforms state-of-the-art methods across multiple metrics, achieving improvements of up to 38.80\%. These results underscore the effectiveness of the TOGCL framework for precise temporal QoS prediction.