Federated Reconstruction: Partially Local Federated Learning

TL;DR

Federated Reconstruction is a scalable, model-agnostic framework for partially local federated learning that balances privacy, communication efficiency, and client heterogeneity, demonstrated through empirical results and real-world deployment.

Contribution

It introduces the first model-agnostic framework for partially local federated learning suitable for large-scale training and inference.

Findings

Outperforms existing approaches in collaborative filtering and next word prediction.

Successfully deployed at scale in a mobile keyboard application.

Provides an open-source library for evaluation.

Abstract

Personalization methods in federated learning aim to balance the benefits of federated and local training for data availability, communication cost, and robustness to client heterogeneity. Approaches that require clients to communicate all model parameters can be undesirable due to privacy and communication constraints. Other approaches require always-available or stateful clients, impractical in large-scale cross-device settings. We introduce Federated Reconstruction, the first model-agnostic framework for partially local federated learning suitable for training and inference at scale. We motivate the framework via a connection to model-agnostic meta learning, empirically demonstrate its performance over existing approaches for collaborative filtering and next word prediction, and release an open-source library for evaluating approaches in this setting. We also describe the successful deployment of this approach at scale for federated collaborative filtering in a mobile keyboard application.