Private Federated Learning in Gboard

TL;DR

This paper discusses Gboard's implementation of federated learning, differential privacy, and secure aggregation to train ML models on user data while preserving privacy, and explores future enhancements like trusted execution environments.

Contribution

It introduces Gboard's specific privacy-preserving techniques for federated learning, combining DP-FTRL and secure aggregation, with strategies to ensure high utility and formal privacy guarantees.

Findings

Effective privacy-preserving ML training on user devices

Strong differential privacy guarantees achieved

Potential for further privacy improvements with trusted execution environments

Abstract

This white paper describes recent advances in Gboard(Google Keyboard)'s use of federated learning, DP-Follow-the-Regularized-Leader (DP-FTRL) algorithm, and secure aggregation techniques to train machine learning (ML) models for suggestion, prediction and correction intelligence from many users' typing data. Gboard's investment in those privacy technologies allows users' typing data to be processed locally on device, to be aggregated as early as possible, and to have strong anonymization and differential privacy where possible. Technical strategies and practices have been established to allow ML models to be trained and deployed with meaningfully formal DP guarantees and high utility. The paper also looks ahead to how technologies such as trusted execution environments may be used to further improve the privacy and security of Gboard's ML models.