PCL: Prompt-based Continual Learning for User Modeling in Recommender Systems

TL;DR

This paper introduces PCL, a prompt-based continual learning framework for user modeling in recommender systems, which uses prompts to preserve knowledge and adapt to new tasks effectively, addressing catastrophic forgetting.

Contribution

The paper proposes a novel prompt-based continual learning approach for user modeling that leverages position-wise and contextual prompts to mitigate forgetting and improve task adaptation.

Findings

PCL outperforms traditional models in continual learning scenarios.

Prompt-based methods effectively reduce catastrophic forgetting.

Experiments demonstrate improved user representation quality.

Abstract

User modeling in large e-commerce platforms aims to optimize user experiences by incorporating various customer activities. Traditional models targeting a single task often focus on specific business metrics, neglecting the comprehensive user behavior, and thus limiting their effectiveness. To develop more generalized user representations, some existing work adopts Multi-task Learning (MTL)approaches. But they all face the challenges of optimization imbalance and inefficiency in adapting to new tasks. Continual Learning (CL), which allows models to learn new tasks incrementally and independently, has emerged as a solution to MTL's limitations. However, CL faces the challenge of catastrophic forgetting, where previously learned knowledge is lost when the model is learning the new task. Inspired by the success of prompt tuning in Pretrained Language Models (PLMs), we propose PCL, a Prompt-based Continual Learning framework for user modeling, which utilizes position-wise prompts as external memory for each task, preserving knowledge and mitigating catastrophic forgetting. Additionally, we design contextual prompts to capture and leverage inter-task relationships during prompt tuning. We conduct extensive experiments on real-world datasets to demonstrate PCL's effectiveness.