Incremental user embedding modeling for personalized text classification

TL;DR

This paper introduces an incremental user embedding method using transformer encoders to dynamically update user profiles, improving personalized text classification accuracy in social media data.

Contribution

It presents a novel incremental user embedding approach that effectively integrates recent interaction histories for personalized classification tasks.

Findings

Achieved 9% relative accuracy improvement over baseline.

Achieved 30% relative accuracy improvement over baseline.

Effective in modeling user histories for personalized tasks.

Abstract

Individual user profiles and interaction histories play a significant role in providing customized experiences in real-world applications such as chatbots, social media, retail, and education. Adaptive user representation learning by utilizing user personalized information has become increasingly challenging due to ever-growing history data. In this work, we propose an incremental user embedding modeling approach, in which embeddings of user's recent interaction histories are dynamically integrated into the accumulated history vectors via a transformer encoder. This modeling paradigm allows us to create generalized user representations in a consecutive manner and also alleviate the challenges of data management. We demonstrate the effectiveness of this approach by applying it to a personalized multi-class classification task based on the Reddit dataset, and achieve 9% and 30% relative improvement on prediction accuracy over a baseline system for two experiment settings through appropriate comment history encoding and task modeling.