One-Shot Federated Learning with Classifier-Free Diffusion Models

TL;DR

OSCAR introduces a classifier-free diffusion model approach for one-shot federated learning, significantly reducing communication costs and eliminating auxiliary classifier training, while achieving superior performance on benchmark datasets.

Contribution

This work presents OSCAR, a novel classifier-free diffusion model method for one-shot federated learning that removes the need for auxiliary classifiers and reduces communication overhead.

Findings

Outperforms state-of-the-art on four benchmark datasets

Reduces communication load by at least 99%

Eliminates auxiliary classifier training at clients

Abstract

Federated learning (FL) enables collaborative learning without data centralization but introduces significant communication costs due to multiple communication rounds between clients and the server. One-shot federated learning (OSFL) addresses this by forming a global model with a single communication round, often relying on the server's model distillation or auxiliary dataset generation - mostly through pre-trained diffusion models (DMs). Existing DM-assisted OSFL methods, however, typically employ classifier-guided DMs, which require training auxiliary classifier models at each client, introducing additional computation overhead. This work introduces OSCAR (One-Shot Federated Learning with Classifier-Free Diffusion Models), a novel OSFL approach that eliminates the need for auxiliary models. OSCAR uses foundation models to devise category-specific data representations at each client which are integrated into a classifier-free diffusion model pipeline for server-side data generation. In our experiments, OSCAR outperforms the state-of-the-art on four benchmark datasets while reducing the communication load by at least 99%.