Efficient Multi-task Prompt Tuning for Recommendation

TL;DR

This paper introduces MPT-Rec, a two-stage prompt-tuning framework that enhances multi-task recommendation systems by improving generalization to new tasks, reducing training costs, and maintaining high performance.

Contribution

The paper proposes a novel prompt-tuning approach that disentangles task-specific and shared information, enabling efficient transfer to new tasks with minimal retraining.

Findings

MPT-Rec outperforms state-of-the-art multi-task learning methods.

It maintains comparable performance while significantly reducing training parameters.

The framework improves training efficiency by up to 90%.

Abstract

With the expansion of business scenarios, real recommender systems are facing challenges in dealing with the constantly emerging new tasks in multi-task learning frameworks. In this paper, we attempt to improve the generalization ability of multi-task recommendations when dealing with new tasks. We find that joint training will enhance the performance of the new task but always negatively impact existing tasks in most multi-task learning methods. Besides, such a re-training mechanism with new tasks increases the training costs, limiting the generalization ability of multi-task recommendation models. Based on this consideration, we aim to design a suitable sharing mechanism among different tasks while maintaining joint optimization efficiency in new task learning. A novel two-stage prompt-tuning MTL framework (MPT-Rec) is proposed to address task irrelevance and training efficiency problems in multi-task recommender systems. Specifically, we disentangle the task-specific and task-sharing information in the multi-task pre-training stage, then use task-aware prompts to transfer knowledge from other tasks to the new task effectively. By freezing parameters in the pre-training tasks, MPT-Rec solves the negative impacts that may be brought by the new task and greatly reduces the training costs. Extensive experiments on three real-world datasets show the effectiveness of our proposed multi-task learning framework. MPT-Rec achieves the best performance compared to the SOTA multi-task learning method. Besides, it maintains comparable model performance but vastly improves the training efficiency (i.e., with up to 10% parameters in the full training way) in the new task learning.