Federated Learning for Mobile Keyboard Prediction

TL;DR

This paper demonstrates that federated learning can effectively train language models on mobile devices for keyboard prediction, improving privacy and prediction recall without exporting sensitive data.

Contribution

It introduces a federated learning framework for on-device language model training, showing improved prediction recall and enhanced user privacy compared to server-based methods.

Findings

Federated learning achieves better prediction recall than server-based training.

Training on-device enhances user privacy by not exporting sensitive data.

The approach is feasible for real-world mobile keyboard applications.

Abstract

We train a recurrent neural network language model using a distributed, on-device learning framework called federated learning for the purpose of next-word prediction in a virtual keyboard for smartphones. Server-based training using stochastic gradient descent is compared with training on client devices using the Federated Averaging algorithm. The federated algorithm, which enables training on a higher-quality dataset for this use case, is shown to achieve better prediction recall. This work demonstrates the feasibility and benefit of training language models on client devices without exporting sensitive user data to servers. The federated learning environment gives users greater control over the use of their data and simplifies the task of incorporating privacy by default with distributed training and aggregation across a population of client devices.