Efficient User Sequence Learning for Online Services via Compressed Graph Neural Networks

TL;DR

This paper introduces ECSeq, a graph compression-based framework that enhances user sequence representation learning for online services, significantly improving efficiency and scalability while maintaining high prediction accuracy.

Contribution

ECSeq is a novel unified framework that integrates graph compression techniques into relation modeling for user sequences, enabling efficient and scalable learning without modifying pre-trained models.

Findings

ECSeq achieves high efficiency with minimal training time.

Inference time remains extremely low at $10^{-4}$ seconds per sample.

Improves LSTM prediction accuracy by approximately 5%.

Abstract

Learning representations of user behavior sequences is crucial for various online services, such as online fraudulent transaction detection mechanisms. Graph Neural Networks (GNNs) have been extensively applied to model sequence relationships, and extract information from similar sequences. While user behavior sequence data volume is usually huge for online applications, directly applying GNN models may lead to substantial computational overhead during both the training and inference stages and make it challenging to meet real-time requirements for online services. In this paper, we leverage graph compression techniques to alleviate the efficiency issue. Specifically, we propose a novel unified framework called ECSeq, to introduce graph compression techniques into relation modeling for user sequence representation learning. The key module of ECSeq is sequence relation modeling, which explores relationships among sequences to enhance sequence representation learning, and employs graph compression algorithms to achieve high efficiency and scalability. ECSeq also exhibits plug-and-play characteristics, seamlessly augmenting pre-trained sequence representation models without modifications. Empirical experiments on both sequence classification and regression tasks demonstrate the effectiveness of ECSeq. Specifically, with an additional training time of tens of seconds in total on 100,000+ sequences and inference time preserved within $10^{-4}$ seconds/sample, ECSeq improves the prediction R@P$_{0.9}$ of the widely used LSTM by $\sim 5\%$.