Vertical Federated Continual Learning via Evolving Prototype Knowledge

TL;DR

This paper introduces V-LETO, a novel vertical federated continual learning method that uses evolving prototypes to transfer knowledge across tasks, effectively reducing forgetting and improving performance in privacy-preserving settings.

Contribution

The paper proposes an evolving prototype knowledge approach and a model optimization technique to enhance continual learning in vertical federated learning, addressing catastrophic forgetting.

Findings

V-LETO outperforms state-of-the-art methods by 10.39% in CIL tasks.

V-LETO outperforms state-of-the-art methods by 35.15% in FIL tasks.

The method effectively mitigates forgetting of previous task knowledge.

Abstract

Vertical Federated Learning (VFL) has garnered significant attention as a privacy-preserving machine learning framework for sample-aligned feature federation. However, traditional VFL approaches do not address the challenges of class and feature continual learning, resulting in catastrophic forgetting of knowledge from previous tasks. To address the above challenge, we propose a novel vertical federated continual learning method, named Vertical Federated Continual Learning via Evolving Prototype Knowledge (V-LETO), which primarily facilitates the transfer of knowledge from previous tasks through the evolution of prototypes. Specifically, we propose an evolving prototype knowledge method, enabling the global model to retain both previous and current task knowledge. Furthermore, we introduce a model optimization technique that mitigates the forgetting of previous task knowledge by restricting updates to specific parameters of the local model, thereby enhancing overall performance. Extensive experiments conducted in both CIL and FIL settings demonstrate that our method, V-LETO, outperforms the other state-of-the-art methods. For example, our method outperforms the state-of-the-art method by 10.39% and 35.15% for CIL and FIL tasks, respectively. Our code is available at https://anonymous.4open.science/r/V-LETO-0108/README.md.