Learning Transferrable Parameters for Long-tailed Sequential User Behavior Modeling

TL;DR

This paper introduces a novel framework for long-tailed sequential user behavior modeling that transfers knowledge from popular users to tail users, improving performance and addressing cold-start issues.

Contribution

It proposes a gradient alignment optimizer and adversarial training scheme for effective knowledge transfer in long-tailed user behavior data, adaptable to various models.

Findings

Outperforms state-of-the-art baselines on four real-world datasets.

Effectively addresses cold-start and long-tail distribution challenges.

Enhances model performance for tail users through transfer learning.

Abstract

Sequential user behavior modeling plays a crucial role in online user-oriented services, such as product purchasing, news feed consumption, and online advertising. The performance of sequential modeling heavily depends on the scale and quality of historical behaviors. However, the number of user behaviors inherently follows a long-tailed distribution, which has been seldom explored. In this work, we argue that focusing on tail users could bring more benefits and address the long tails issue by learning transferrable parameters from both optimization and feature perspectives. Specifically, we propose a gradient alignment optimizer and adopt an adversarial training scheme to facilitate knowledge transfer from the head to the tail. Such methods can also deal with the cold-start problem of new users. Moreover, it could be directly adaptive to various well-established sequential models. Extensive experiments on four real-world datasets verify the superiority of our framework compared with the state-of-the-art baselines.