TransFR: Transferable Federated Recommendation with Adapter Tuning on Pre-trained Language Models

TL;DR

TransFR introduces a federated recommendation approach leveraging pre-trained language models and adapter tuning to enhance transferability, personalization, and privacy in federated learning settings.

Contribution

The paper proposes TransFR, a novel federated recommendation framework that combines pre-trained models with adapter tuning for improved transferability and privacy.

Findings

TransFR outperforms existing federated recommenders in transferability.

Adapter tuning enhances personalization and privacy in federated settings.

Theoretical analysis confirms benefits of adapter tuning for effectiveness and privacy.

Abstract

Federated recommendations (FRs), facilitating multiple local clients to collectively learn a global model without disclosing user private data, have emerged as a prevalent on-device service. In conventional FRs, a dominant paradigm is to utilize discrete identities to represent clients and items, which are then mapped to domain-specific embeddings to participate in model training. Despite considerable performance, we reveal three inherent limitations that can not be ignored in federated settings, i.e., non-transferability across domains, ineffectiveness in cold-start settings, and potential privacy violations during federated training. To this end, we propose a transferable federated recommendation model, TransFR, which delicately incorporates the general capabilities empowered by pre-trained models and the personalized abilities by fine-tuning local private data. Specifically, it first learns domain-agnostic representations of items by exploiting pre-trained models with public textual corpora. To tailor for FR tasks, we further introduce efficient federated adapter-tuning and test-time adaptation mechanisms, which facilitate personalized local adapters for each client by fitting their private data distributions. We theoretically prove the advantages of incorporating adapter tuning in FRs regarding both effectiveness and privacy. Through extensive experiments, we show that our TransFR model surpasses several state-of-the-art FRs on transferability.