Federated Learning in Practice: Reflections and Projections

TL;DR

This paper reviews the progress, challenges, and future directions of Federated Learning, emphasizing real-world applications, privacy guarantees, and emerging trends like large models and heterogeneity.

Contribution

It proposes a redefined FL framework focusing on privacy principles and suggests leveraging trusted execution environments and open-source tools for future development.

Findings

FL has scaled to millions of devices with meaningful privacy guarantees

Real-world systems from major tech companies demonstrate FL's practical viability

Key challenges include verifying privacy guarantees and managing device heterogeneity

Abstract

Federated Learning (FL) is a machine learning technique that enables multiple entities to collaboratively learn a shared model without exchanging their local data. Over the past decade, FL systems have achieved substantial progress, scaling to millions of devices across various learning domains while offering meaningful differential privacy (DP) guarantees. Production systems from organizations like Google, Apple, and Meta demonstrate the real-world applicability of FL. However, key challenges remain, including verifying server-side DP guarantees and coordinating training across heterogeneous devices, limiting broader adoption. Additionally, emerging trends such as large (multi-modal) models and blurred lines between training, inference, and personalization challenge traditional FL frameworks. In response, we propose a redefined FL framework that prioritizes privacy principles rather than rigid definitions. We also chart a path forward by leveraging trusted execution environments and open-source ecosystems to address these challenges and facilitate future advancements in FL.