Modulated learning for private and distributed regression with just a single sample per client device

TL;DR

This paper introduces a novel privacy-preserving federated learning method that enables devices with only one data sample to collaboratively learn accurate models by transforming and sharing private data representations.

Contribution

It proposes a new approach that injects calibrated noise and shares transformed data samples instead of model updates, suitable for single-sample devices.

Findings

Achieves unbiased gradient updates comparable to centralized training.

Enables privacy-preserving learning with extremely limited local data.

Outperforms traditional federated learning in single-sample scenarios.

Abstract

This work focuses on the question of learning from a large number of devices with each device holding only a single sample of data. Several real-world applications exist to this one sample per client setup up including learning from fitness trackers, data/app usage aggregators, body-worn sensing devices, and daily event monitors to name a few. When a client has only one sample, the standard federated learning paradigm breaks down as a local update based on that single point is far from being useful, especially in the earlier rounds for estimation of the model coefficients. This utility is further weakened by the privacy-inducing noise applied at every round. This work caters to this problem to enable such clients to collaboratively contribute to effectively learn a global model without leaking the privacy of their data. The proposed approach injects a single, carefully calibrated noisy perturbation to transform the sample at each client, followed by a post-processed representation which is shared with the server. These representations aggregated at the server are processed to obtain an unbiased gradient update that in expectation matches the non-private centralized gradient while preserving data privacy. This approach is different than traditional private federated learning, where the communication payloads involve model coefficients as opposed to privately transformed data samples. This method enables devices with extremely limited data to collaborate and learn accurate, privacy-preserving models without requiring large local datasets or sacrificing individual privacy.