Federated Continual Learning for Text Classification via Selective Inter-client Transfer

TL;DR

This paper introduces FedSeIT, a federated continual learning framework for text classification that selectively transfers knowledge between clients, improving performance while preserving privacy in heterogeneous task environments.

Contribution

It proposes a novel federated continual learning method with selective inter-client transfer and domain overlap assessment for NLP tasks.

Findings

Achieved an average of 12.4% performance gain over baselines.

First application of federated continual learning to NLP.

Effectively mitigated inter-client interference in heterogeneous tasks.

Abstract

In this work, we combine the two paradigms: Federated Learning (FL) and Continual Learning (CL) for text classification task in cloud-edge continuum. The objective of Federated Continual Learning (FCL) is to improve deep learning models over life time at each client by (relevant and efficient) knowledge transfer without sharing data. Here, we address challenges in minimizing inter-client interference while knowledge sharing due to heterogeneous tasks across clients in FCL setup. In doing so, we propose a novel framework, Federated Selective Inter-client Transfer (FedSeIT) which selectively combines model parameters of foreign clients. To further maximize knowledge transfer, we assess domain overlap and select informative tasks from the sequence of historical tasks at each foreign client while preserving privacy. Evaluating against the baselines, we show improved performance, a gain of (average) 12.4\% in text classification over a sequence of tasks using five datasets from diverse domains. To the best of our knowledge, this is the first work that applies FCL to NLP.