Active Federated Learning

TL;DR

This paper introduces Active Federated Learning, a method that intelligently selects clients based on current model and data to reduce training iterations by up to 70% without sacrificing accuracy.

Contribution

It proposes a novel client sampling scheme conditioned on model and data, significantly improving training efficiency in federated learning.

Findings

Reduces training iterations by 20-70%

Maintains model accuracy with fewer rounds

Mimics known resampling techniques under certain conditions

Abstract

Federated Learning allows for population level models to be trained without centralizing client data by transmitting the global model to clients, calculating gradients locally, then averaging the gradients. Downloading models and uploading gradients uses the client's bandwidth, so minimizing these transmission costs is important. The data on each client is highly variable, so the benefit of training on different clients may differ dramatically. To exploit this we propose Active Federated Learning, where in each round clients are selected not uniformly at random, but with a probability conditioned on the current model and the data on the client to maximize efficiency. We propose a cheap, simple and intuitive sampling scheme which reduces the number of required training iterations by 20-70% while maintaining the same model accuracy, and which mimics well known resampling techniques under certain conditions.