Mix2FLD: Downlink Federated Learning After Uplink Federated Distillation With Two-Way Mixup

TL;DR

Mix2FLD introduces a communication-efficient, privacy-preserving federated learning framework that combines uplink federated distillation with downlink model updates using two-way mixup, improving accuracy and convergence under channel asymmetry.

Contribution

It proposes a novel Mix2FLD framework that integrates federated distillation and federated learning with two-way mixup to enhance efficiency and privacy in asymmetric channels.

Findings

Achieves up to 16.7% higher test accuracy.

Reduces convergence time by up to 18.8%.

Effective under asymmetric uplink-downlink channels.

Abstract

This letter proposes a novel communication-efficient and privacy-preserving distributed machine learning framework, coined Mix2FLD. To address uplink-downlink capacity asymmetry, local model outputs are uploaded to a server in the uplink as in federated distillation (FD), whereas global model parameters are downloaded in the downlink as in federated learning (FL). This requires a model output-to-parameter conversion at the server, after collecting additional data samples from devices. To preserve privacy while not compromising accuracy, linearly mixed-up local samples are uploaded, and inversely mixed up across different devices at the server. Numerical evaluations show that Mix2FLD achieves up to 16.7% higher test accuracy while reducing convergence time by up to 18.8% under asymmetric uplink-downlink channels compared to FL.