Feature-based Federated Transfer Learning: Communication Efficiency, Robustness and Privacy

TL;DR

This paper introduces feature-based federated transfer learning, significantly reducing communication costs, enhancing robustness against data issues, and addressing privacy concerns through theoretical analysis and experimental validation.

Contribution

It proposes a novel feature-based approach that lowers communication overhead and analyzes its robustness and privacy aspects, which are less explored in existing federated learning methods.

Findings

Reduces uplink payload by orders of magnitude

Demonstrates robustness against packet loss and data issues

Provides privacy leakage analysis and mitigation strategies

Abstract

In this paper, we propose feature-based federated transfer learning as a novel approach to improve communication efficiency by reducing the uplink payload by multiple orders of magnitude compared to that of existing approaches in federated learning and federated transfer learning. Specifically, in the proposed feature-based federated learning, we design the extracted features and outputs to be uploaded instead of parameter updates. For this distributed learning model, we determine the required payload and provide comparisons with the existing schemes. Subsequently, we analyze the robustness of feature-based federated transfer learning against packet loss, data insufficiency, and quantization. Finally, we address privacy considerations by defining and analyzing label privacy leakage and feature privacy leakage, and investigating mitigating approaches. For all aforementioned analyses, we evaluate the performance of the proposed learning scheme via experiments on an image classification task and a natural language processing task to demonstrate its effectiveness.